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fc. sK2;^3S<tH?*'^ai^n/fctfift(oni3iM(oi^iis 

[tt5tt3S2] ±gB2i>:?S<tH?*^^l±l$n/cRihti, 
[fgsRJSS] ±IBtTi;jfai«¥Sli> ±fB 1 i^^S^tH? 

«i m 5 nrctTiii i: oeajstt© i: fcsBti-r 5 c t ^iRf^ 
m^m4^ ±iBt7iDiBis*Stt, ±m:k^itm^ 

m 5 nfctf K) * T tc ±fB 2 :^^3S{kH^ 3b^«ltil * n/c 
tTS);&j/-'yi/xtT®)i: LTlB1S-rsci:^!|^S![i:t-S 
%mm 3 a E«cO a 4^ -y b 

Cli*«5] ±fB^Ti<i^*^*S^±, ±iB->— 'i^vxt? 

[IS*3S6] ±fB l'>:?S{kH?A^^til$n^i:, ±iB 

±HEtTi<) ffi > ±is - v xtTifi fc S n 5 ^ 
?Tii^|ilfctilJi$-tt, ^OS^^, ±iB 1 '^?i{tH?*^^ 
ttisnfc^-&fc{i, ±ISfTifilB1i^S:«, ±fB->-'5r>' 
Xtfift^rSS^LTIBItL, ±gB 1 -^^iftH^ A^^ttJ $ n 

a^A^^i±i5nssT'±iB>'-'5ri/xtTi!)fc^*n«># 

*JS4lB««0n>-}-:-y h^Bo 
immm 7 ] ±IB A^j¥S*^A:'3 L /c:m^<ongi?1t$fi 
i:±BBtT»)iB^#ia*^iB1SLfc±tB 1 :^?i<kB^^*^^m 

ffi^SC t^1f®i:-r-5.ti5j<« 1 IEi(<Oa#-y h^^o 
[sS*3l8] ±iEA:'3#ISA^A:^L/'c#^1t««:^F 
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^(Or]T>' Ki:±fERiftIB1t^S7b^fBtiLfc±IE 1 

i;«:!|^®i:-r«>ti*JS7lB^<oa5ft«y hSSo 
[li*3l9] ±tBm^i03Ti/Ka. ±iE 1 ?>:?^<I:H 
? A^^m ^ n/ctfefc A:^i ^ n/c t ©T'^ c 
■rSii^3S8tBic<Oa.-p>y h^go 

±ii^Pmmic J; •? ISiJ^ nfc±fBi>'-'5r >Xtf IftfcBII 
t>^m^(0 n V > K i: ±fBtf »)IB^¥IS*^IE« L fc±fB 

, Sfc^{i;^^ci;;&!|tMi:-r-Sfi*Ja4fBic<OD4<-y h 
±iBtfifiSB«¥©t±, ±tBtTK)^^^- Kfc*3i.^T±fB 

Cli^iSl 2] ±fBm^«3V>Ktt, ±iEt7i«l¥S 
■t- F i: SnrciSttfc A^'jSn/c t OT'feS c i:;Sr!^m 

±iE2i>j?S{t:H^^^*- Ffc*3l.^T, ffi^co5^a51t$R 
*^A;']^nfci^fc±fB 1 i^^ikB^A^I^m^n^i:, s 
Ki^gPlf $B*^±fB 2 :^j>JS{tH? i: 5 c t ^I^Sl ^;•r 
L%mm 1 4 ] ffiiic'ongPltffiA^A:'^ S nfc?tfc±lE 
fc. ^^i^ngI5tt?fiA^±IB2i>:?S{kH^i:Snsci:^!|t 

mt-r^m^m i iBiconij-c-v f^m„ 

[ii5t?3S 15] ±tB 1 i^^^S^tia? SLt>*±IB 2 :i^?S{tH 
^{±, ±IEA:'3#S*^A:^ L/-c^^1t?Sr'*5 c fc^lt 

^Ti)^Dtti^l^«I^IfcS•^^'■'T. —nVjWi^'ik^'t^ViWlk 
±IEtTit)St^XieT'8li^$tv/c:tTi&fc»^('^T. ±tBi() 

itEAs/^^SA^A:*] L/-c±IBi1-a51f ISA^6 2 ^>J?S(tH? 
> «r«llii-rs2i>:^{t:H^«liUxSi:. 
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±s2tf idSfc^xST-ti, ±12 2 i>:®ftH? *^«tni ^ 
±f2tfttf2'^xigT'ti. ±12 1 i^^{tH^*^«^m?ns 

c»5R« 17] ±12 2 rkmitm^if^^m ■^ntz.nm 
{±. ±12 1 i^?s^tla^A^^ti3^nfc^fift^cs5$t:-o^T 

im^mx 8] ±IBtTi/)IStiX?iT'ti:. ±12 1 
H?Jt)^^l±lSn/-ctTift*^ SiSffi!lHijti:±l22;^3iftS 

s n/tff ») t <ommL<D t, i: ict 2M* n^ct 

im^m 1 9 ] ±l2tTi!)l2tiXgTHi. ±12 1 'Xmt 
H?*^«Sm5nfc:?Tt!lST'»c±f2 2 :^^?a<tH^A^«Sttl^ 

Ho 

immm 21] ±12 1 miim^iimm-$n?>t. ± 

±l2tTfiftt±l:'3XigT't±> ±IB~>-'^>xtTi!)ii:#sns 
^tfiftjb<)illctli3i!$n. ^(OSam. ±12 1 :^^{bH?*< 

l^m^n/c^-^tca, ±f2fT®)gEtiXllT'ti:. ±IE>'- 
':r>xffi/)^fii^LTf21i.L, ±H2 1 i^^SitS^Jb^l^to 
*n^cA^o/c^-&tc(±. ±feiTl)!)m;'axST'«. ±121 

[fSSiJl 2 2 ] ±f2X:/:^©*^A:'a Lf=m^©^1.a5lt 

^tb$nfc^fifii;^M31-^ttT^^J211-r5M^J21iX 
g^W-r?.c t^l^ltii-rSfi^JS 1 6f2i8<Oa4^<y h 

[fi^^a 2 3 ] ±t2A;']#SA^Ati L fc^^^mm^m 

±s2)i3®f21SxeT'ti. ±s2^^S{iatc J; 0 n/t 
m^«3v;x Fi:±l2tTiftl2«Xier-f21t^ti/-c±B2 l 

^Ct^^mtt^i^T^mz 2H2M«Oa.-}-:'y h^gOtT 

[li^^ 2 4 ] ±i2mS<D n F « , ±12 1 i^I^Sit 
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Cl«*«2 5] ±l2A;'7¥IS7b^A:^Lrc^^tS$B%^ 

±ia^^saati: J; o igMsn/-c±i2i^-^^ >'xtTiijtcM 

S m^iO n -7 > K i: ±l2f7il!)f 21«xaT'f 21«$ nfc± 

t2>'-'5r>X?Ti6i:^gaiS-^(tTS;®tB«-rS3ffiSf2M 
Xigi:^^-r^ci:^!|tMi:-r5W*« 1 9f2«oa.-p 

±IBtTi(jl2tiXg-??t±. ±f2fTlft^S^- K{C*5V^T± 

12 1 '^3i{kH^*^«lt±52ns fc. < t Sl^ 1 :^>:3a 

m^mz 3f2ic(ODd?-y h^HtOtTS/)¥S75ffio 
[M^^ 2 7 ] ±f 2m^<0 n >' K t± , ±f 2tf ift^S 

2 8] 2 :^mtm'i^^m-^-r^rzit>(D z 
±H2 2 3^?S<tH?^^^- Flciot/^T. em^o^i-gpiffs 
Kii-asif IB*^±t2 2 :^>:3S{tH^ i: ^txS c i: ^mmtir 

[i« jkjs 2 9 ] t£m.(DnmMmti'^x:h $ nfcigtc±i2 
Mfc-rswsR^i 6B2«g«on4?-y h&m<onm^my5 

50 3 0] ±12 1 ^X^itm'FRZf±nE 2 ^^Si^kH 

mt-t%m^m\ 6mm<^o^^-y vm.m<r>'um^^ii 

[tt^]S3 1] i!){lFg|5^$i|fflibTfi»l^l±S3K*-<±«a 
^X^i:. 

±H2tTl/)i*^X^T-?^^$n/cffiJjtC»-:5t/>T. ±I2®I 

40 thtlXgi:, 

ngP1f $a>&A;^i-r5 A;':¥IS*^A;^; U/-c±f2ngB1t^A^ 

«ltiSXSi:, 

±f2A:'3#S*'A:'3 L/-c±i2ngP1tfB*>6 2 ;>;?i<tH? 

*^tiif ^ 2 i*3ifkH?i^mxsii, 

±H2tf iftj^^xsT-s^^ ^ nmmmi.'m.t ^ nmmm 

±mmik^r.mr'ii. ±12 2 i^?iftH?A^^tb ^ 

50 iS466n, 
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^i^m-r^ 2^^?iit:s^^^dHxii^. 

±fBtTi^i*^XgT^i*^$nfcfT»l^lB1lt-;g)tTiftfati 
XS^^Wb. 

±HBtTifti*^X?iTti. ±IB2;^?SftH^/)^1^to^n^ 

Sic. ^^2:^j^?iibH^?b^i^ai^n/cfTiftcoto3Egiiaip]:6^ 
±iBtTiftfBigxs*t?ti. ±$i\:k^imTtmm'^n^ 

[000 1] 

^^■r^^.1-^y hSScOtfKj^S^^iS. D4<»y hS^tO 

y n y ^ A(7)iBii^ ntc zfa L.tmrnwicm'T^ o 

[0 0 0 2] 

[0 0 0 3] ( 1 ) mmt. wmm(onmf3^^mm(o 
[0 0 0 4] (2) ^^^mmmmoM-rrcisb. mm;i 
^\ 

[0 0 0 5] (3) nm-r^mc^i^'^n^m^mm 
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[0 0 0 6] c(D^r>^jim^ic^o. ^<(Dmmmi±^ 
[0 0 0 7] mmcnW}^m.^^rcisb(oy5mat. ft« 

W^t^O^bT. -t-rU^^^ (Modelling) . 

>y (Ruring) > ^^r^^^UV^' (Capturing) . ^ 

^f—zy3> (Imitating) ^n^e^mi 4 

[0 0 0 8] ^-r'UV^ti. ^<co:;^cof5)v>^?b^lStTt* 
20 [0 0 0 9] i!)JBgi:coi^a6^^Slfl*A'«^ 

v^i^.>&i^v>T^T-'u>'^''^sifmbTi/^^o mmmi^^i. u 
[0 0 10] ^^':f^^rvy^it. ia^bfc^^^u >y 

^tc. ccOTjffiti. i!jft<OSfg6^^tTli!J^fUfflb/c 
^cOTS§/c:46. aJlt6^^Kjf^-^^bv^ffijf^^S;^^C 

[0 0 1 1] ^^x-s/a^ti. i!j1?gtcijl|^^i:|plblft 
[0 0 12] 

[58HflA'«^}*bcJ:a ^-r^PS] hc5T% ifi^. 
^^b/c?g«c^^nfca.i<*y hgg. I^t5f^§^^yha 
4-:'y h/)^^g^^nTl^^o C^Ocfc^JSrn.K-y h^Sti. 
-flS^Stc:JoV>TglW^n^:^-^}aii:(a/cfl54^^Wb. 
50 :x— »f (^v^i) rnn<j mx^^ tl^-Dtc 



-4- 



7 

[0 0 13] C<DX d y^'^-y hmmtmm(Om^(0^ 

[0 0 14] m^i^£. mm(omviditci^'r^<Dtmmc^ 

[0 0 1 5] L;?)^b. ±aiL/ccfc^^^[^.iOi!l*^tCtTitl 

[0 0 16] ±'^Lrc^'rvy^^it. mmmc 

[0 0 17] Sfc. ±a!L/'c;l/TU>'^li. a4-^y hX 

[0 0 18] ±aib/c4^-v:/^-vU>yti> cco^rffi^ 

fflV^T. n4t'>y hSB;?)Mi^ ^^ilbfc 

COO 1 9] _ha!L/c:-Y^f— >3>ti> fi?»J^(i\ TY. 
Kuniyoshi et a I. /Learning by watching: Extracting 
reusable tasl< knowledge from visual observation of 
human performance. IEEE Transactions on Robotics 
and Automation, 10(6) 1799-822, 1994J ^tcfBSc^nr 
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[0 0 2 0] CO<i;3 3S:?*3l^0^1ttC^^T 

^Ci:tOT'#^n4-:^y h^S. COct: a ^n.i-^-y h^g 

[00 21] 

/ci6tc. 2ls:5IWtc1^;g>a4<^y h^H*i> ifii^giS^^SiJSS L 

[0 0 2 2] ccT\ 2:^^?sfba^7b^i^m^n/ctTi& 

30 ti. i^^!S^bH^*^1^^m^n/'ct7»tcS§*-t?o?fifiT 
[0 0 2 3] ccD<k-9^a*'-y h^fi«. tTifio^^ 

[0 0 2 4] ^fc. ±3ab/c:eW«:Mfi!ct-Sfc«!)tc, * 

timm Bli-r s tTiftiBitie t^mi^. ±tmmm 
50 2 i^c?aftB?*^^m^ n/'ctTii!i<ow]mi6]A^iSi6 e.n. 



-5- 



9 

tt^nsc bTi/^-So 

[0 0 2 5] ccr\ 2:k^itm'^timm-snrznm 

[002 6] C(D.t^;S:a4-*y h^^<Otfi!)^^73ffiT- 

[0 0 2 7] $fc. ±^Lrcsm^mfS.-r^tcii>ic. * 
^■^ntc 1 i^^fbH^^^imfs 1 :^^?a{tH?«imxig 

[0 0 2 8] ccT% 2:k^itm7-fi'^^tii^nrcnm 

[0 0 2 9] CtOcfc^^ad-C-y h^geOtTift^Wya^* 

[0 0 3 0] ^tc. ±^i.rcEff^^m^t^rciibi>c. * 
^)!)<Dt^^ffilfi]^cs■:5^^T. -n'fsmnim-^-t^'fimm^ 

XSi:. ±fBiTlfiS^^XiST'i*SSnrc?7ifttcS-ift,> 

«:«IUi-rs U^!MfbH?«imxiei:. ±taA:'3¥ia*^A 
:^ L/£±lBngi51S$BA^6 2 i^iJ?i{tB^ ^l^tB-r ^ 2 
fbH^^^tiiXgi:. ±fBtTiiift^Xgr-ft^$nfct7ift 
«:BB1t-r-&tfi!)tEtixei:^^L, ±fBtf ilJi^^XgT- 



(6) 2003-39363 

/O 

ti. ±fB2:^^®{tEg?*^«m^ni.«tc. mmZ'Xmt 

IBttX^THi. ±fBl'^?S{kH^*^«im5n^i:, '>=S: 

< tt^^ 1 ^^?s^tH^7b<^m^n/-c^TSJ*^tB^^ns 

8B^?nfct>OT?S5o 

[00 3 1] C C T\ 2 i^?S<tH^*^1^t±i5nfctf IfJ 

JO [0 0 3 2] c(D^o^j:ytjy^L.%mmwiCBm^n 

CO 0 3 3] 

[0 0 3 5] ( 1 ) ^mm(ojBmic:^^f^^^-y hmm 

^*nPgl5:i:::^y h 3 A. 3B. 3C, 3 D;b'^ilJ5^n^ 
30 tmc. BittgprL- ^y h 2<DHUiggPSQl^iggPtC^n^* 

[0 0 3 6] ^<*g[5:x-^y h 2tCti:. m Z I^Cjjk-t ^ 
tc. CPU (Central Processing Unit) 10. DRA 
M (Dynamic Random Access Memory) 1 1 . y^^yi^zL 
ROM (Read Only Memory) 1 2. PC (Personal Com 
puter) F-r >^->^x-XIiI£Sl SRlfim^mm 
iHlSSl 4;(3^^l^g|5^^X 1 5^/rLTtiStC^^$n^C^ 
ICcfc om^^n/cn >hD-;l/gi5i 6 i:. cou^^^y h 

§o S/c. H§»gi5:3.-^y h 2tcti. Ov-fx^y hSH 1 (0[p] 

[0 0 3 7] ^fc. !SgP:3.-^y h 4tcti. nSPcOt^Si^ 
mt-^^/ci^^OC C D (Charge Coupled Device) 
"yZOh. :x— Hf*>6cD rftT^;5j ^ rpn<j ^l'^'::>/c 

^y^^-t^+hz 1 my5K\iLm'^^mw^x^(Dmm^m 
s-r^/ci6cogggi-fev*t2 2 ngp^^m^-r^/c46 
50 co*T^^o4-x>2 3 i:. mt^mo:>mw^^ti~t^rzib 
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LED (Light Emitting Diode) t^Eff^ 
[0 0 3 8] ^PgPZL- h 3 A~3 DCO^gn 

53^. ®gpa- ^y h 4St;flHf*gPrL- «y h 2 cOMl^gPiJ^. 

^n'en^^m^i^(Dr<7^=LX.-^2Sx. 252. 

• • • St;4"'-rVi^3;<— ^2 6 1 . 262. • • •A'^ 
BB^^nri^^o T^^^X-^2 5i. 2 5 

2. • • • ti. -9— 4<^-^^«^^bT^LTV^^o 
+)--4<^-^<?:)|Kii!j(cj;0. Pgp:x- ^y h 3 A- 3 Djb^ 

[0 0 3 9] -^-LT. cn6ft3IJS•fe>+^ 1 8. ^31* 
-trv-t^ 1 9> ^^y^-tr>+^2 1 . g6gf*tr>t^2 2. "^-T 
^D.tx>2 3. Xlf-;?7 2 4St5S'1^-r>>^3y-^2 

61, 2 6 2. • • ' r^}i(D^m'^y^ms\zLE^R 
t/^r^^ax— ^ 2 5 1 , 2 5 2. • • • ti. -en^* 
nw/s-r^^Ny^2 7 1 ~2 7 n^frLT3F>hD'-;i/a5 

1 eOfi^jaSlslKl 4 ^S3g?^n. CCD:^p<^2 0 

Rr>v^yx'j 1 i\it. ^n^*nmmmM^ \ a hum 
[0 0 4 0] m^wmm^ \ Aht. ±a?to#-tr>-9*A^^ 

DRAMi \{H(DFR'^iiLmicm^A^m'r^o ^/c. fi^ 

SaSlslliSl 4ti. cnhtttC/^^yT^U 1 7*^6««&^n 

cn^DRAMl 1 [^tOpjT^ffiHtclS*ft-r^o 
[00 4 1] CCOct-ptcLTDR AM 1 1 rttC*S*rt^n 

U^Bx-^ti. clCD^^C P U 1 0;?)'«c:cDD>i<^y h^W. 

[0 0 4 2] ^^±C P U 1 Olft. ajp^y hS§ 1 

:7^>yi/ n-ROM 1 2tClS*rt^n/'cSiJSiiyuiJ/^A^P 
C:^-K-r>^--7x-Xlp]g§l 3^/M,TX(iItg^S! 

^mb> cn^DRAMi 1 tc^ssfl-r^o 

[0 0 4 3] ^fc. CPU10t±> C(Om±^^(O^'0{C 

m^mmm^ i 4 ^ o d r am i 1 um^k^m^n^^ 

[0 0 4 4] cpuioti. ccDpm^^iSLrj 
DRAMi \{z^mi.rzmwyci^*^Lsicm^\^^rm< 

T^^zlX.-^ 2 5 I , 2 5 2. ' ' ' ^mwj'^'^^c 

^tccfco. m^=L^-y h A^±-riE^{cmib^rco. k 
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[0 0 4 5] S/c:> CCD^C P U 1 0\t. jJ^gtC^SCT 
B^(l^<^LTXt!"-;^2 4(c-^^^c^tc:j;0> 

[0 0 4 6] CC0ci:-9tcbT. Ccod4< *y h^B nc*3 
70 ffllt^!)Ht^^:/SbTS»6q^^:^TfJb^#§<fc'5^^:^;^nTV^ 
[0 0 4 7] (2) MSUT'n^'^^iACOy h'^x7<ll;S 

3tCfel/^r. -T-VWX- F'^-T/^ • b'-r-\'3 Oti. ceo 
fflffljyn^^i:.<OSTfiStcffiSb. ^JS^^^-rV^^X • 
K^-r>'^:6^e)^^-rVWX • F^'-T/^ • ir^y h 3 1 t!)^?) 

^^^nri/^^o cotf-a-. ^T-v<;^x • F-^-r>'^ti. 

CCD:^;^^20 (1^12)-^. ^-T vl|cOil#tOn Vtf 
20 ^-^TfflV^^nS/N-F^i^xriCfig^T'^^^-bXf §c 
^^f^^n/c:t:/v>x^ FTfeD. ^^j5-r§/N-K»>x 

[0 0 4 8] ^tc. ^y ^ • ^--/K • ;4-yi^x^ 

h3 2ti:. T-VWX- F^-f/N • U-r-\'3 OcOSTteS 

5i. 2 5 2. • • • ^tOv'N— F^^xTtcr ^-fex-r^ 
/ca6(0-r>'^-77x-x^Ji«-r^v'7 h»i7xr?¥T^ 

;g>/^'— ^-v;!/ • Pvl^'^y F 3 3 

F»^xrs¥T3^§^^'7-"^:t^-v^-v 3 a 
30 {fi(Da>e?(0'r>'Wx • F^^>'^^ga-r^v>^ F^ 

xTPTJ^^T-V^-rx • F-^'-Y^^ • V^> — >>> 3 5 
a 4< -y h 1 (Oait^ Wa-r ^ V :7 F X T p-^:^ ^ 
T^-9-V^^F • D.-K^y F 3 6 ^^^^filiS^nTl^So 
[0 0 4 9] -T^^-i^'^ • ^:^^>x^ F 3 7ti. ^:/>>' 
x^ F • v^-v 3 8St/+>--t:X • v:t>— 3 9 

:6^^1t^^nTl>^o :ty^v^x^ F • 

i^>3 8ti. a^^x^^y^ • +^->'^ • :^:/v^x^ F 3 

2. ^ F;i/ • »^xr • ix-r-V4 0. ;!S^D'ryuy-i/3 
> • u^-v 4 1 tc^sn^#v:7 h »^xT?¥cDeaw 

40 T^'mmt^VyV^x.T^X&K>. -9— lfX•V^.- 
v^>39ti. ;^^U;^-F2 8 (112) tc^g^^n^cn 
^^^>-3 >':7 7''Y;l/tC|Ba^^nTl/^^§:ty^v^x^ F 
cOg^^1t$8tcao\^T^;^yi>'x ^ FOfi^St^BS-r^ 

[0 0 5 0] ^ F;1/ • '^xT • l^^-V4 0(i:. a4<7^^ 
^y ^ • -9-— • ^:/>^x^ F 3 2co±fi@tcfitBb. lii 
^SaS-»^^^l^j5!iii^^(7)iiOa4<*y F^@ l <oS*W^ 
«iflE^}l«'r^V:7 F*^xyP;6^6m^^nT«.-^o S 
/c. T^Uy— ^"3 > • U-r-\'4 Ui. ^F;l/-»^xT 
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[0 0 5 1 ] ^*3. ^ F;l/ • "^xr • i^-r-\'4 omST 
yu^— v'a^- U'<ir4 1 0*f*6^?5:y37 hr>xrill 

^^ii4jc^-ro 

[0 0 5 2] ^ K;V • "^xT • U'Y-^4 Otc4ol/>rt±. 

ttjffl. ^mmmm. ^m^mm. ^^^tHffl> ^^y^-t 

;l/5 0-5 8Mt>lcA:^Jirv>7'^ ^^r>^ 
:x-;l/5 9:^^^W"r^|giS^6 0 ^li^-^y^'r^ 

[0 0 5 3] ^>^^6 0cO^fi^®gi-tS^'^-;l/5 0- 
5 8ti. ^y ^ • -y-— • ^T'v^x^ h 3 ZCDv'^ 

— n.-n'^y h3 3tCcfcODR AM 1 1 (MZ) 

^tf. • n4< y h 3 3ti:. m^Oilfl^i^tc 

[0 0 5 4] A:^-tv>"r^^xn>/^— ^-tv^^-;!/ 
5 9ti. Ctl^ft^ffiil^v^:^.— ;V5 0 — 5 8:6^^^;^ 

3>-u^-v4i (1^3) tcm^^-r^o 

[0 0 5 5] > • u-r-\'4 1 ti. mb\z 

[0 0 5 6] tTii!)^f';bv-ry^U 7 OtCti. ^6tC^ 

■Tcfc-^tc. r/vyT-u5^a;5v>;^<;^^/-i^>g.j ^ re 

T. ^n^'nffiS:L/ctfK}^-r;U7 0 1 - 7 0 n /)"^^tt 
[0 0 5 7] ^Lr. cn^?T®I-tT=^;l/7 0 1 — 7 0 n 
5 9A^e>IS^Sm?b^^^6n/'c^^'^. gftO^iJj^m 50 
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tc^s i;Tma?o<j: 9 ic.®it^:-r ;i/ 7 3 Kun^^ nri/^^ 
[0 0 5 8] r^^. c.(r>nm(omm<Dm'^. §?fKj^r-r 

;i/7 0 1 -7 Onti. :k(onm^\k'&'t^^mii\^X. 
0 7tC^*rck^^ lOtOy-K (t^S) NODEo-N 
ODEn;^)^^fficO^(?)y — KNODEo—NODEntC 
M^*r^/)^^^y- KNODEo-NODEn CDF^^^ 

.^•r^T-^ AR c 1 ~AR Cn tc^^LT^'n-rns^ 

[0 0 5 9] SftWtC. §fTift^7'>'l/7 0 1 - 7 0 

nti. ^n^'nsacotf®)^7^>'^7 0 I ~7 0 n 

r^y-FNODEo— NODEn tC^n^'tlWlS^-^ 
T. cne>y-FNODEo~NODEnStCEI8tC^ 

[0 0 6 0] CC0m^S^S8 OT^i:. ^coy— Ftc:fo 

[0 0 6 1] x^tzti^^x. ms<r)^mm^m.Boxm.-^ 
n^y-FNODE 1 0 o-eti. r4<-;i/^^^to (ba 
LL) J hh^omMA^Wiif^?Lhnrcm^\c. ^^mw^ 

#^m^«tC^^8n^^^04'^-;^^0 TA^^ (S 1 Z 

E) J tf^ ro;b^6 1 oooj coiGHTfe^c^^. rpg 

mVQ^^^ (OBSTACLE) J ^ IS^^^^'^-^ 
m^^X<D mm (D I S T ANC E) J TO*^^ 1 

ooj (Dmrnx^^^hm^o^j—YKm^-r^tc^^o^ 

[0 0 6 2] ^tc. Ccoy-FNODEi o o THi. 
-7 0 n ;b^M)^6^tc#fiS-r^^1f ^'r;b7 3Rt>'*H6^ 

•r;i/7 4tc^n^'n{SJt^n/c^tsii&Q'#®iAt^o>'^^ 
y-^Mod-^. .®if^f^;i/7 3tc^^$nfc r^t>* 

(JOY) J . (SURPRISE) J gL<(i 

rggb^ (SADNESS) J ov^-rn7b^o^^5'y-^ 
fi7()^ rs 0*^6 1 OOJ (owmx^^t^m^<Dy-Y 

tC S 5^ S C h ^ ^ cfc -9 t C ^ r:) T V ^ ^ o 
[0 0 6 3] S/c. miSI^^8 0Tti. Htoy-F 

^offii^5i^j (TDffiitc^ott^ rs^$ty-Fj cofTtc^ 

toy-FNOD Eo-NOD E n/)>e3gj^T'^^y-F 

«:6^^UHB^nTi/^^^«tci. rA;^j-r-<y h^j . 

/-c h t icm^x t s m<o^y - f n o d e o 
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-FNODEo—NODEn {dm^-r^WAcmtjir^^ 

gi^eS^J (;DlffltC4ott§^tT^Om^O?Pti 10 0 [%] 

[0 0 6 4] Lfcfi'^'Dr. msco^mm^ms or^m^ 

(BALL) J L. ^(0^^-?V<0 (S I Z 

E) J r o^^e 1 0 0 OJ OSBHT'fe^^V^^Ig^Jg 

^f^^^.^p,nrcm^itat^ rao [%] j ost^T ry 

— KNODEi 2 0 (node 120) J ^tOi; 

# TACT I ON 1 J (onW}tf^\tiii^n^<^tt^ 
[0 0 6 5] ^?7®)^7^;i/7 0 1 — 7 On ti. ^n^*n 

C Ocfc ^ 8 0 LTlBiS^ n/c y - F N o 

DEo-NODE nff^l^<'^^mf:i'^^<^o\zLrmfS.^ 

KNODEo— NODEn (OViWM^^ 8 0 ijffl L T 

[0 0 6 6] m5ic7rs'rnm^m'e>^:jL-)V7 \ n 
ifi^f'y^^-r'y^u 7 o<?:)#fTSj-^:'r>'i/7 o i — 7 On 

^JlHfitO^SV^fT»J^-r/^7 0 1 - 7 On^^^tti^^J^nZ-c 
T> cn^tTlfi::3*T>Khv^'5o ) F;l/ • "^^r • 

6tC^l/^TTffiJtCgfiB^n/c:fTi!)^f-Vl'7 0 i ~7 0 n 

[0 0 6 7] ^fc. nm^m'eiyzL-?v7 u*. nm^ 

L/cCh^^ig't>^i-;l/7 2. ^tt^7^;l/7 3&t>'** 

E[l+l]=E[t]+ke xAE[t] 



76 



[0 0 6 8] =¥S-t>?n^-;l/7 A:^-trT> 
[0 0 6 9] ^LT. ^g^v^^~;l/7 2(i. C(Dmm 
^^ZT'yV 7 OtCfctt^tTift^x;l/7 0 1 ~7 OnO?* 

[0 0 7 0] ffi;^. ^.lt^x;i'7 3ti. r^t/ (jo 
y) J . r^.L^ (sadness) J . r^O (anger) J . 
(surprise) J . mm (disgust) J Rtf TS 

n (fear) J cD^lt 6 ocDtBifttcov^T. ^tSlft C'^tc 

[0 0 7 1] Mi^mclt. .^1S^T-^;b7 sti. A:;^-tr*^ 
M«gm-i:> ^tO^^cOOvK-y FSB 1 OtTiftt. fj[ilM 

[t] . ^ffifD^CDltllftcO/'^-^;<-^fa^E [t] . 

30 (Dmwxomm^m'ri^m^ k e ^ lt. ( i ) s:tc <j: o 
T:^^(OjiWtc:fctt^-^toltilji7)/^-^y-^fiiE [t4- 
1] ^wihL. cn^jgsEo^coitiijfo/^vy-^ffiE 
tt] tm^i^^^^v\cLr^(omm(D/^'^?^-^m 

[0 0 7 2] 
C3S1] 

- • (1) 



[0 0 7 3] ^4b\ #igM«S«^m:^j-trv>T--i'^xr3 
v/^— ^-tv^i— ;b6 8 3?j^6coaa]3b^§1til!i<?)^^^y — 

^fio^i&flAE [t] {c^commmm^^^^Mt^ 

mti r^Oj otii!i(0/^^^-t$fffl«o^»fiA E [t] 

[00 7 4] CCT. tiMi't^y'r^c ^7.:jy/^—^'e 
v?^— ;l/6 8:6^e>031^hti. l^t5f^^tT®jco:7 >r — F 



40 1t$BT-feO. iS«^-r;l/7 3ti. dcDJ^-p^lffBtCcfco 

Ttg^lt^^^t^-^^o cnti. fi^ij^tf. rp:^^^j t 

7 2lctA:^^nT:teO. ^^^i/'^->'l/7 2Ci. ^^D 

mmcmr5\.^x^um'^'f)\/7 o i — 7 on to^^fjs-r^s 

[0 0 7 5] tTi/}^m<^>^^-F/^«y ^ti. tTl/3 

50 cfcD^^n^^OT'^-i^T^cfcU^o 
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[0 0 7 6] —-n. *HE^T=";b7 4 ti. ^mW^ (exer 
cize) J . rglt^i (affection) J . ^"^^ (appetit 
e ) J RCF ^^^?^.L^ (curiosity) J OSVHcffljtLfc 

-trvy-r^ ^xn>>'^— ^^v^jz— ;l/5 ^ti'h^T^^t^ 
[00 7 7] *t6^f-Vl/ 7 4 10 

l[k'fl]=l[k]H-lCi xAI[k] 



lcai^n^^<Oi:#tD^oa)(5RcD^ii!l«*A I [k] . 
m&<0^<r>'^iS^o^f^zf:/^-^mit\ [k] . ^(?D®:^cD 
^jg^at-tm^k i i:LT. m^i^WT (2) iC^ffl 
l/^T^^cDjga^tCite^t^^coa^^cO/^^^ — I [k + 

-^fBl [k] ^g^}^^^J:-9tcLT^(Oa^*cO/^^ 

[0 0 7 8] 
[S2] 

• • • (2) 



[0 0 7 9] =^4b\ ^,^Sm&t>'m;^-tW'r^^x:3 
y/s-^^v^ii-yi/e 8jb>e)Oaiaji&h*^^^®:*o/^^ 

;^-^{fi(D^®]«A I [k] tC^'(7)SJgiJ#;&^;^^:6> 

-^fiO^ifiSA I [k] tc:^^%i5#:^^^§J:"5tc 
[0 0 8 0] J^fe. :*:^j!SiiOmitc4ol/^Tti. 

tf^a^^R (*fi6) <o/^^p«-^fiiA'^-^n^n 0:6^6 i o 

OSTOS5H-e^lfit*^cfc^lc*IM^nT*5D. ^/c. 
[0 0 8 1]— 7?^ ^ K;^ • »^xr • u-r-\'4 ocDm:^j 

•V4 1 ^D^7i^J^^^^/^— ^^7 ljb^e>^^6n^ rfff 

^6 9com-r^<i^saa^5^:i-;i/6 i -6 7tc^^ 
[0 0 8 2] ^LTcne><t-^saa^>^^->'i'6 i -6 

^25i~25n (^2) 

Xtf-:^2 4 (02) t^hm^t^^(0^^^-'^ 
mS/lUit r@j COL E Dtc:-^;^^|gii!iT-*-^^^fiE 

^ h 3 2C0/^-^-v;b • P4<^y h 3 3 

1 4 (^2) ;&|li:^^/M.T*tJE'r^T^^aLai-^2 5 

1 -2 5nX(iXfcr-?b2 4Xei:L EDtcm^^iSSdi'r 

[0 0 8 3] c:^0<fc^^cbTa4-^y Hc:Jol/>T 



[0 0 8 4] ( 3 ) *^j5s^?Dm^Tffll^^^TifJ^s^^^7^?^ 

a^LT> (Shaping) ^#tfn^>3&^ 

[0 0 8 5] TB.F.Skinne 
r/'The Behavior of Organisms. ,App I eton Century Cro 
30 f s. New York, NY. , 1938J (CHB«^nTV^;S)B. F. Skinner 

-J ti- h u-::^>f\cm'^\^^rc-^<ommi5m^. i 9 8 

O^^ttc. tTS)±M>L^S^#^OGary Wilkes^. ^ )\^ti 
cODl|l^#T^fe^ Karen PryorTb^^tlBJ-e^R^-tt/co ^ U 
;^ ~ h > ^'THi . fi^ J: ^ ;l/ ;^ cOH/IIMtc ffl 

[0 0 8 6] ^U^y;0-^Jf'r^S<®Jl'^^^fg-r^o 

^ikmmm-3\,^^ § c ^ t 2 :^j^?SfbH^ (^i* tt^^fLS 

?) ir^^o ccD^2 3*<l:S^t±. iftit^tc^oT. ^-5 

50 Lrcmc(o^\:kmtm^^^^^o cnti. mM^os 
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[0 0 8 7] ^^)-jti-V\y-:^y^*^t. 'pr^<h%>^X 
[0 0 8 8] mi XT-^y:/: ^ ^) ti—^^\ — \yT 

mLr§r^\B\. CT^tf 2 o~5 oiHif^if. m^ff^'^o^y 
[0 0 8 9] m2x^-y':f rim^m^^o ^kic. 

[0 0 9 0] msx-r^yy : avy FO^- F^^x.^o 

ttw-^o n-wFy-F^^g-p^-ra^y^i. i/j*^*^^ 

[0 0 9 1] m4X7"'yy : tTlft^^-To ^^tc. ^ML 
[0 0 9 2] t^±ltt9^L/ccfc^tc. ^U'y:^-Fl>-- 

[0 0 9 3] tXTT'ti. cc?D^'J^y:^-hU— :=->yo 
J!^S^ffli/^^Ch-e±aib/ca4<^y h^B 1 Jb^tf®J^¥ 

[0 0 9 4] (4) a.i<'y FSB'N^Ojgffl 

ift Pj^^ MS -r? 1 1 ^ C h t> T # ^ c 1/ ^ ^ ^ ^ ^ 

[0 0 9 5] a>i-^yh^BHi. 13 9 tc^-Ti: -9 (c. A 
timiOlh. 1 :^j^3SitH^1^t±igP 10 2^. 2 

m^mms^ i o 3 i:. nm^k-^^ 1 0 4 vimtmsi^ 

10 5^. tTSjm:^^ 106^. gl^^SJgP 107^. 

iSffJBtigP 1 0 8 h^m^xx^^^o 
[009 6] C ce. A:^a5 1 0 Hi. ^2 tC^f v-r 



(11) 1fP^2 0 0 3-3 9 3 6 3 

^a4>>2 3-^^*y^-b>-9-2 1 TS^o A:^§P 1 0 1 

(i. ^p^fi^xti-tvt^fi^^ 1 ^^SfbH^i^tagp 1 0 

2RD*2:^j^?4ita^jisa5i 0 3tc«^*r^o s/c. a 

1 0 1 (i. ^?^{i^^^^|g^a5 1 0 7 
[00 97] \:kmmi'^^^U\ oz^t. A:^a5io 
s^^i^m-r^o l^^3S^t^^^i:^ ^i6n.-n^y F^Bi 

JO tf. OTj^'^y FSB 1 (Dm^^o:>f±t] (eS^S< /c/c< c 

tt'^mf^n^o iXTx^t. \:!Km\:m'¥'hi.x r^^^^i^ 

-J ^i^'5^^(i^^ffl'/^^o 1 :^i^3S{tH^«lttigI5 1 0 
2ti:. l^^^Sftlll^^l^tB-r^^. 1 ^^?S<t:fl^*?f W 

i^^gp 1 0 4 &t/tTi&HB1tgP 1 0 5 tc{tt$&-r^o 
[0 0 9 8] 2:^3S{tH^M«gl5 1 0 3 ti. A;^gP 1 0 

^i:f«]1itc^i6P>1^^^y FSS 1 tciS^$nTi/>^fecoT* 

^^■r^ii^tcti. n>1<^y F^B 1 ^2:;^^?S^t 

H?^^-t— F ^ 0 10 tc^-r cl: ^ LTS^^tf 

[0 0 9 9] Ell OtC^-Tcfc^tC. z^^k^itm'^^^'^ 

o T ^ m jS: V 
[0 10 0] ^i/^TXT-^y^s 2tc*5i^T. i:^^?S<t:EI 
50 X-r'y:/S 3tc:fei/>T. Xr-^y^SlT-^^ 

[0 10 1] r^^. 2:fm{tm=^<oWL^^mts c<Dm 
tCPg^^ n-r\ ^Jxtf. n.1<>y F^B 1 
tci:j:^3Sfl:S^^^-^;^e>n/c:C^^^g$(lE]. <^y;^tf3 0 

iE]i:^±i^m-r^^. 't(oMm^2^^xmtm'^h\.x%m. 

2^^mia^^^M^ftEtt*^ci^:?.-e#^o C(Dm. 
m?i^t. 2:^^3S{bia^^M:3v>F^^;^^;6^^ttSt?:) 

[0 10 2] ^*3. c:cD2^^?SitH-?ti. a.K'y FSB 
tVX ry^y Fj 1/^*9 g^fi^^fflv^^o 

[0 10 3] z^xi^im'^mm^i 0 3(r^. 2^mitm 

50 [0 10 4] ftKli*^g|5l 0 4(i. ^^y-F («^) 
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^gp 1 0 4 ti. 1 :!m\tm'f^^U 1 0 2 2 ^^?s<t 

B^ji^sSP 1 0 3 tf^h^^^ntc 1 :iKmtm^ms 2 

[0 10 5] tflibtS'S.gP 1 0 5 ti. ?fi!j?3^^a5 1 0 4 
Itgp 1 0 4 ti. 1 :^^?S{l:S^1^ttlg|5 \0 2^xJ2:im 

mf^m^^n. ^fiij^^^- y^^cr^^xt^h i 

[0106] tTi!jm:^gp 1 0 6 ti. nmm-^^ \ ost^ 

3T>FJ&E!4ti:^-rtB:^-trv>-rw' ^Xr3>/^— 20 

[0 10 7] ^^^viigp 1 0 7 ti. XtiUl 0 1 ;«)^e>A 

mt^o ^Wm\C{t. ^nWM^X OTl^t. HMM (HI 
ddenMarkov Model) ^Sffl LTlt/S^nT^o D . HMM 

[0 10 8] ^SEUgPl 0 8ti. ?T»J^^^-FtC55: 

t'^^'X ibnrc-m(ovimh^^wmu \ o 7 xmrn^n 30 

;g>o u^yYhWM-:3^fxmmmm.^nr~m<onm 
i,t. vimt^mM 1 0 4 tcien^n^o 

[0 10 9] D4?^y FSSim±coJ;*5=^1t^^W'r 

ti. lyKmtm^^^x. u^^-yvmm^t^ ^n^x<D 40 

2:^^3S^bH?0#;^e)n/'c-acDtTi&«:tB1I-r§o 

[0 1 10] c(o^im^^:^ti:(D-m(o^m^m \ 1 
T. tTifi^s^-Ftcia^^n^o 

[0 111] y :/S 1 1 tii^oV^r. FTb^e 

y-F-\iOS^5f^;?)^S]^<l:^n. X-r-y^S 1 2tc:fo 5(? 
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[0 112] T.'r-jfS 1 3Tti> Xx'yT'S 1 2^05 

*ijgij^ni)o u^®fbia^*'^^;^6nfc^^tcti. x^ 

^y^S 1 7tC3ityo XT^^y^S 1 3tc4o(/^T. X^^K^t 
H?3b^^;^^nTV^^V>«'&tc(i. Xx^y^S 1 4 tcji 

[0 113] XT^^yT'S 1 4-eti> X'r^yT'S 1 2 T tb 
^y^S 1 2tcHoTSa':^^cDtTitf^i*^'r^o Xx^y >^ 

s 1 4tc*5v^T. 2:^?s^i:ia^*'«-^^6nfc^^tcti. 

Xx^y^S 1 StcittJo 
[0 114] X-r-^y^S 1 STHi. XT'*y:/S 1 2X^ 

tj i.mim^mm(Dnmicm^nw}t Lxmrni^. xt^ 

^y^S 1 6tC3ityo 

[0 1 1 5] XT^^y^S 1 6Xl^t. 2:k^itiSm^'^^X 
LTXT^^yy S 1 2 tcj^^o 

[0 1 1 6] Xx^y^S 1 7Tt±. i:k^itiE^f)^^X 

[0 1 1 7] XT-^y^s 1 STti. \:kmtm^f^'^^x 
^nrcnmt^-m(Dnmti.x. XT^^y^^s 1 ixn 

^ b /c n V > F ^ Bia-^tt Xtm b T^7 f § o 

[0 118] ^mmmmictsf^f^^im^^yjrtiXi^t. 
[0 1 19] r^^. m 1 1 (Dyu—'^^-vx^t^ 

<. D#^y h^Sl;^)^mil«?3tTl)!i^tTofcfflmtc. ^(D 

[0 12 0] s/t. m 1 1 0:7a— ^-v- h•e^i^ xf^ 

-y-fs I 6{c^i^x. 2:k^im^i:i^^xhnrcnm(o 

[0 12 1] ^C5T% illM^ti. 2^^^?SftH^^<$o 

Ta>t:^y h^B 1 ^mm.(DmY^^hmm^^c.hfs^x^ 
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[0 1 2 2] 5t"r 1 ssco:^at^> ^^i^m^mmmmiSL 
[0 12 3] mmm^mt. m^imi^^m (iio. m 

[0 12 4] ^.fRjw#itti. nwi^mc-^-^^mm. m 

[0 12 5] c(Dyjm'Vi±. -B^^f)m^^n^t± 

[0 12 6] :^^tc2S@o:??jSt±> ^tTii!iK<7DM^^^ 

■r^) ^^^O0^5StcE5?iJ'r^C^;6^T*#§o ^1 2 

■To ^/c. ^^^^y h^B 1 Z?'^^ 1 2 tcwe>n/c tflft^ 
t o fcttSI^O-gpjgria 1 3 tc^-To 
[0 1 2 7] 01 2^ffll.^T. fll^^^^ D>-K'yh^S 

fjT^i. 013 (G) i^CTjkir^olC. D^^yhSHUi 

ti. 0 1 2<oy— F [D I G] (O^^tCi^/S-r-So 
[0 12 8] ^Wtc> n>t^'y h^B Hi. 0 13 (A) 

AND] coi^^tcfe^irf^o a>1->>yh^BUi. 9c'r 

mi 3 (B) tc^f j:-5tci^tT^tea6^. -r^tD-^. y 

-F [STAND] (Om^?^^^y-F [WALK] (D^ 

BifCM^^^o cfoy— F [WALK] ^(ommt. m 

m(0^mxh^y-h* [dig] tC3fi-:5< t(?)T'ti:^l/^ 
^y F^B 1 it. SUOff Sj^y- F [STAND] 

*^e>r^<?&"r^o ^5:^% c*;)^.. sttwtcy-F [sta 
ND] (ovimicm^^r>ici.r^^<. s/c. s^bs^ 

[0 12 9] :kica^.-y FSB Hi. 0 13 (D) tc^ 
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-r^dicm^. -r^^^y-F [s it] {cm^-r^o 
c<oy-F [s I T] ^(ommt. mm(ovimxs>^y 

-F [D I G] tcifi-:5< tcoTfe^/ci6. DW^ti. 
r^-^vFj ^HV\ u4^^'y F^BUC«UT2:>^?S{tia 

mmff^m^si'tEn. n^K^yF^sui. 2^^?s{tia^o# 
^ibntc^mx^^m^nm (y-F [s i t] ) 

[0 13 0] SSl^rn^-^ y F^BUi. 013 (E) ic 

JO ^-Tcfc^tc. 2 2^cOHf)KJ-Cff Lif&i6^o -r^tD-^y— F 
[PUSH] tcM^-r^o L^^L. C^oy-F [PUS 

H] ^cDM^ti. miiom^T^^^y- F [d i g] 

:^l>o ^cea.l^^y hSB Hi. ^UOfiift^y-F [S 

I T] <?Dmi7b^e>p?flji&-r^o ccomc^. ^mm^y- 

F [SIT] Ot^filtc]^^cl:dti:UTtc}:<. ^tc. M 

[0 13 1] l^l^TD4<>y hSBl ti. y-F [H E L 
LO] icM^i.. 013 (F) tc^^cfc^tcfcHuP^± 

mmuc ry^y Fj ^ai\ 2:^^5S<bH^^-^^^o 
[0 13 2] l:^±cr)c^:o^iW^^^>t^y FSBi 
^C>^D^ST. a>i<'y h^B 1 :6V- F [DIG] tc 

-J tm\^\ u^-p^y F^B 1 tcMLT 1 ^^SiftH^^-^ 

^i>o coi^^sfifbs^ti. u^.yhmmif^'^pfimcon 

[S I T] - [H E L LO] - [D I G] ^V^d — 

30 mcDnmrnm-t^o 

[0 13 3] J^fc. ±a>bfc [SIT] - [HELL 

o] - [D I G] ti^ommcm^-f. mcomm^i^-z^ 

V^^o m^l£. 013 (C) tc^-TJ:-?^ 

SE^tm ^^t>"^y-F [KICK] ^/rLTn>1t*^y 

[0 13 4] mm(Dnmfjmn^tii:k^im^^^^ 
^t. mm^it. ini^tcffifi^giji-::5^tt^ziv>'F7- 

40 F^n.-p'y hmm 1 tc-^-^^o ^O^.. P>i<^y FSB l 

;b^^n'T> F7- F^es^Hcs^bfcc t^mm-r^fcisb 
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PROBLEM TO BE SOLVED: To efficiently learn complicated and continuous 
actions. 

SOLUTION: In a robot device, a command for executing the action determined 
stochastically in an action determining part 104 is outputted by an action output 



part 106 based on the action. When approaching desired action, a trainee gives 
secondary reinforcing factor to the robot device to use it as a mark for 
approaching the desired action. When the secondary reinforcing factor Is 
detected, the action determining part 104 changes transition probability to the 
desired action. When the robot device performs the desired action finally, the 
trainee gives primary reinforcing factor to the robot device. When a series of 
actions reaching the desired action are related to commands, a series of actions 
read from an action storage part 105 and a command recognized in a voice 
recognition part 107 are associated and stored in an associative storage part 
108. 
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CLAIMS 
[Claim(s)] 

[Claim 1] An action decision means to be robot equipment which a right hand 
side is controlled [ robot ] and makes action appear, and to opt for action of 1 
based on the appearance inclination of action, The above-mentioned right hand 
side is controlled based on the action for which the above-mentioned action 
decision means opted. An action output means to make the action concerned for 



which it opted appear, and an input means to input external information, A 
primary reinforcer detection means to detect the primary reinforcer beforehand 
set up from the above-mentioned external information which the 
above-mentioned input means inputted, A secondary reinforcement factor 
detection means to detect a secondary reinforcement factor from the 
above-mentioned external information which the above-mentioned input means 
inputted. It has an action storage means to memorize the action for which the 
above-mentioned action decision means opted. The above-mentioned action 
decision means It is robot equipment with which the appearance inclination of 
action for the secondary reinforcement factor concerned to have been detected 
whenever the above-mentioned secondary reinforcement factor was detected is 
raised, and the above-mentioned action storage means is characterized by 
memorizing the action by which the primary reinforcer concerned was detected 
at least if the above-mentioned primary reinforcer is detected. 
[Claim 2] The action by which the above-mentioned secondary reinforcement 
factor was detected is robot equipment according to claim 1 characterized by 
being action until it results in the action by which the above-mentioned primary 
reinforcer was detected. 

[Claim 3] The above-mentioned action storage means is robot equipment 
according to claim 1 characterized by memorizing the action by which the 



above-mentioned primary reinforcer was detected on the basis of relevance with 
the action by which the above-mentioned secondary reinforcement factor was 
detected before the action concerned. 

[Claim 4] The above-mentioned action storage means is robot equipment 
according to claim 3 characterized by memorizing the action by which the 
above-mentioned secondary reinforcement factor was detected by the action by 
which the above-mentioned primary reinforcer was detected as sequence action. 
[Claim 5] The above-mentioned action decision means is robot equipment 
according to claim 4 characterized by raising the appearance inclination of each 
action included in the above-mentioned sequence action. 

[Claim 6] If the above-mentioned primary reinforcer is detected, it will be made 
as [ reproduce / the above-mentioned sequence action / in order ]. The 
above-mentioned action output means When each action included in the 
above-mentioned sequence action is made to appear in order, consequently the 
above-mentioned primary reinforcer is detected When the above-mentioned 
sequence action is decided and memorized and the above-mentioned primary 
reinforcer is not detected, the above-mentioned action storage means The 
above-mentioned action output means is robot equipment according to claim 4 
characterized by reducing the number of each actions included in the 
above-mentioned sequence action until the above-mentioned primary reinforcer 



is detected, and making it appear in order. 

[Claim 7] Robot equipment according to claim 1 characterized by having the 
associative storage means which associates and carries out associative storage 
of the action by which the above-mentioned primary reinforcer which the 
predetermined external information and the above-mentioned action storage 
means which the above-mentioned input means inputted memorized was 
detected. 

[Claim 8] It is robot equipment according to claim 7 which is equipped with a 
speech-recognition means carries out speech processing of the speech 
information which the above-mentioned input means inputted, and recognize as 
predetermined language, and is characterized by for the above-mentioned 
associative-storage means to associate and carry out the associative storage of 
the action by which the above-mentioned primary reinforcer which the 
predetermined command and the above-mentioned action storage means 
recognized by the above-mentioned speech processing memorized was 
detected. 

[Claim 9] The above-mentioned predetermined command is robot equipment 
according to claim 8 characterized by being inputted after the above-mentioned 
primary reinforcer is detected. 

[Claim 10] The robot equipment according to claim 4 characterized by to have 



the associative-storage means which associates and carries out the associative 
storage of the above-mentioned sequence action which a speech-recognition 
means carries out speech processing of the speech Information which the 
above-mentioned input means inputted, and recognize as predetermined 
language, and the predetermined command and the above-mentioned action 
storage means in connection with the above-mentioned sequence action 
recognized by the above-mentioned speech processing memorized. 
[Claim 1 1] It is robot equipment according to claim 8 with which it has the action 
learning mode for learning action, and the above-mentioned action storage 
means is characterized by memorizing the action by which the primary reinforcer 
concerned was detected at least if the above-mentioned primary reinforcer is 
detected in the above-mentioned action learning mode. 

[Claim 12] The above-mentioned predetermined command is robot equipment 
according to claim 11 characterized by being Inputted immediately after 
considering as the above-mentioned action learning mode. 
[Claim 13] Robot equipment according to claim 1 which will be characterized by 
using the external information concerned as the above-mentioned secondary 
reinforcement factor if the above-mentioned primary reinforcer is detected after 
having the secondary reinforcement factor setting mode for setting up a 
secondary reinforcement factor and Inputting the external Information of 



arbitration in the above-mentioned secondary reinforcement factor setting mode. 
[Claim 14] Robot equipment according to claim 1 which will be characterized by 
using the external information concerned as the above-mentioned secondary 
reinforcement factor if that the above-mentioned primary reinforcer was detected 
continues more than the count of predetermined after the external information of 
arbitration is inputted. 

[Claim 15] The above-mentioned primary reinforcer and the above-mentioned 
secondary reinforcement factor are robot equipment according to claim 1 
characterized by being the speech information which the above-mentioned input 
means inputted. 

[Claim 16] The action decision process of being the action study approach of the 
robot equipment which a right hand side is controlled [ robot ] and makes action 
appearing, and opting for action of 1 based on the appearance inclination of 
action, The above-mentioned right hand side is controlled based on the action 
for which it opted at the above-mentioned action decision process. The primary 
reinforcer detection process of detecting the primary reinforcer beforehand set 
up from the above-mentioned external information which an input means to input 
external information as the action output process of making the action 
concerned for which it opted appearing inputted. The secondary reinforcement 
factor detection process of detecting a secondary reinforcement factor from the 



above-mentioned external infornnation which the above-mentioned input means 
inputted, It has the action storage process of memorizing the action for which it 
opted at the above-mentioned action decision process. At the above-mentioned 
action decision process The action study approach of the robot equipment 
characterized by memorizing the action by which the primary reinforcer 
concerned was detected at least if the appearance inclination of action for the 
secondary reinforcement factor concerned to have been detected is raised and 
the above-mentioned primary reinforcer is detected at the above-mentioned 
action storage process whenever the above-mentioned secondary 
reinforcement factor is detected. 

[Claim 17] The action by which the above-mentioned secondary reinforcement 

factor was detected is the action study approach of the robot equipment 

according to claim 16 characterized by being action until it results in the action 

by which the above-mentioned primary reinforcer was detected. 

[Claim 18] The action study approach of robot equipment according to claim 17 

that action by which the above-mentioned primary reinforcer was detected is 

characterized by what is memorized by the basis of relevance with the action by 

which the above-mentioned secondary reinforcement factor was detected before 

the action concerned at the above-mentioned action storage process. 

[Claim 19] The action study approach of the robot equipment according to claim 



18 characterized by memorizing the action by which the above-mentioned 
secondary reinforcement factor was detected at the above-mentioned action 
storage process by the action by which the above-mentioned primary reinforcer 
was detected as sequence action. 

[Claim 20] The action study approach of the robot equipment according to claim 

19 characterized by raising the appearance inclination of each action included in 
the above-mentioned sequence action at the above-mentioned action decision 
process. 

[Claim 21] If the above-mentioned primary reinforcer is detected, it will be made 
as [ reproduce / the above-mentioned sequence action / in order ]. At the 
above-mentioned action output process When each action included in the 
above-mentioned sequence action appears in order, consequently the 
above-mentioned primary reinforcer is detected When the above-mentioned 
sequence action is decided and memorized at the above-mentioned action 
storage process and the above-mentioned primary reinforcer is not detected The 
action study approach of the robot equipment according to claim 19 
characterized by reducing the number of each actions included in the 
above-mentioned sequence action at the above-mentioned action output 
process until the above-mentioned primary reinforcer is detected, and appearing 
in order. 



[Claim 22] The action study approach of the robot equipment according to claim 
16 characterized by having the associative storage process which associates 
and carries out associative storage of the action by which the above-mentioned 
primary reinforcer memorized at the predetermined external information and the 
above-mentioned action storage process that the above-mentioned input means 
inputted was detected. 

[Claim 23] The action study approach of the robot equipment according to claim 
22 which has the speech-recognition process which carries out the speech 
processing of the speech information which the above-mentioned input means 
inputted, and recognizes as predetermined language, and is characterized at the 
above-mentioned associative-storage process by to associate and carry out the 
associative storage of the action by which the above-mentioned primary 
reinforcer memorized at the predetermined command and the above-mentioned 
action storage process recognized by the above-mentioned speech processing 
was detected. 

[Claim 24] The above-mentioned predetermined command is the action study 
approach of the robot equipment according to claim 23 characterized by being 
inputted after the above-mentioned primary reinforcer is detected. 
[Claim 25] The action study approach of the robot equipment according to claim 
19 characterized by to have the associative-storage process which associates 



and carries out the associative storage of the above-mentioned sequence action 
memorized at the speech-recognition process which carries out speech 
processing of the speech information which the above-mentioned input means 
inputted, and recognizes as predetermined language, and the predetermined 
command and the above-mentioned action storage process in connection with 
the above-mentioned sequence action recognized by the above-mentioned 
speech processing. 

[Claim 26] The action study approach of the robot equipment according to claim 
23 characterized by memorizing the action by which the primary reinforcer 
concerned was detected at least if it has the action learning mode for learning 
action and the above-mentioned primary reinforcer is detected in the 
above-mentioned action learning mode at the above-mentioned action storage 
process. 

[Claim 27] The above-mentioned predetermined command is the action study 
approach of the robot equipment according to claim 26 characterized by being 
inputted immediately after considering as the above-mentioned action learning 
mode. 

[Claim 28] The action study approach of the robot equipment according to claim 
16 which will be characterized by using the external information concerned as 
the above-mentioned secondary reinforcement factor if the above-mentioned 



primary reinforcer is detected after having the secondary reinforcement factor 
setting mode for setting up a secondary reinforcement factor and inputting the 
external information of arbitration in the above-mentioned secondary 
reinforcement factor setting mode. 

[Claim 29] The action study approach of the robot equipment according to claim 
16 which will be characterized by using the external information concerned as 
the above-mentioned secondary reinforcement factor if that the 
above-mentioned primary reinforcer was detected continues more than the 
count of predetermined after the external information of arbitration is inputted. 
[Claim 30] The above-mentioned primary reinforcer and the above-mentioned 
secondary reinforcement factor are the action study approach of the robot 
equipment according to claim 16 characterized by being the speech information 
which the above-mentioned input means inputted. 

[Claim 31] The action decision process of being the action learning program of 
robot equipment which a right hand side is controlled [ learning program ] and 
makes action appearing, and opting for action of 1 based on the appearance 
inclination of action, The above-mentioned right hand side is controlled based on 
the action for which it opted at the above-mentioned action decision process. 
The primary reinforcer detection process of detecting the primary reinforcer 
beforehand set up from the above-mentioned external information which an 



input means to input external information as the action output process of making 
the action concerned for which it opted appearing inputted, The secondary 
reinforcement factor detection process of detecting a secondary reinforcement 
factor from the above-mentioned external information which the 
above-mentioned input means inputted, It has the action storage process of 
memorizing the action for which it opted at the above-mentioned action decision 
process. At the above-mentioned action decision process Whenever the 
above-mentioned secondary reinforcement factor is detected, the appearance 
inclination of action for the secondary reinforcement factor concerned to have 
been detected is raised. At the above-mentioned action storage process The 
action learning program of the robot equipment characterized by memorizing the 
action by which the primary reinforcer concerned was detected at least if the 
above-mentioned primary reinforcer is detected. 

[Claim 32] It is the program documentation medium by which the action learning 
program of robot equipment which a right hand side is controlled [ learning 
program ] and makes action appear was recorded and in which computer control 
is possible. The action decision process that the action learning program of the 
above-mentioned robot equipment opts for action of 1 based on the appearance 
inclination of action, The above-mentioned right hand side is controlled based on 
the action for which it opted at the above-mentioned action decision process. 



The primary reinforcer detection process of detecting the primary reinforcer 
beforehand set up from the above-mentioned external information which an 
input means to input external information as the action output process of making 
the action concerned for which it opted appearing inputted. The secondary 
reinforcement factor detection process of detecting a secondary reinforcement 
factor from the above-mentioned external information which the 
above-mentioned input means inputted, It has the action storage process of 
memorizing the action for which it opted at the above-mentioned action decision 
process. At the above-mentioned action decision process Whenever the 
above-mentioned secondary reinforcement factor is detected, the appearance 
inclination of action for the secondary reinforcement factor concerned to have 
been detected is raised. At the above-mentioned action storage process The 
program documentation medium by which the action learning program of the 
robot equipment characterized by memorizing the action by which the primary 
reinforcer concerned was detected at least if the above-mentioned primary 
reinforcer is detected was recorded. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the program documentation 
medium by which the action study approach of the robot equipment which acts 
autonomously, and the robot equipment with which such robot equipment learns 
desired action, the action learning program of robot equipment, and the action 
learning program of the robot equipment concerned were recorded. 
[0002] 

[Description of the Prior Art] Conventionally, breaking is performed in order to 
tame animals, such as a dog. For example, the owner of a dog teaches action of 
a "hand", "stability", etc., etc. to a dog. However, the owner of many dogs has 
made the mistake of ordering coincidence, making it attach a dog to a desired 
location. For example, a trainer depresses the hips of a dog toward the ground, 
repeating the word "stability." It is from the following reasons that this approach 
is not successful. 

[0003] (1) An animal has forced the selection which pays a trainer's words or 
either of the learned behavior attention. 

[0004] (2) In order to repeat an instruction two or more times, an animal does not 
understand the part of a throat for whether it is connected with action before long. 
[0005] (3) Before acting, it is ordered in very many cases. For example, since 



"stability" is ordered while the animal still stands, "stability" cannot be related 
with the condition of having sat down. 

[0006] For such a reason, many trainers teach an instruction and action 
separately. Action is taught first in fact and an instruction is taught below. 
[0007] There are modeling (Modelling), a lure ring (Ruring), a capture ring 
(Capturing), an imitation (Imitating), etc. in the approach for teaching action to an 
animal as a typical thing, and it has the description as shown in drawing 14 , 
respectively. Hereafter, it explains briefly. 

[0008] Although modeling is an approach which the owner of many dogs tries, 
that the trainer who is an expert carries out never does not have it. This 
approach will praise, if an animal can be physically manipulated to a desired 
position and that position can be reached. By this approach, it is an animal in the 
passive state. Therefore, in many cases, the efforts of action study do not bear 
fruit. Moreover, it is not easy to realize the complicated motion of the more than 
attained to a quiescence position by this approach. 

[0009] RUARINGU is similar with modeling except for a point without physical 
contact with an animal. A trainer can place a toy and food before the nose of an 
animal, and can guide an animal to a desired position using this. Although a 
result with this sufficient approach in the case of an actual animal is obtained, 
that use is limited to the time when a certain position and very simple actuation 



are taught. 

[0010] Unlike the modeling and RUARINGU which were mentioned above, a 
capture ring uses spontaneous action of an animal. For example, whenever a 
trainer recognizes it as the animal taking lessons from the desired position, or 
having taken exact action, he gives positive reinforcer to an animal. Although the 
result in which it is good also about this approach in the case of an actual animal 
is obtained, he needs to wait until an animal attaches a trainer to a desired 
position. For example, when teaching "stability", a trainer needs to wait for an 
animal to sit down spontaneously. Moreover, this approach is difficult to teach 
continuous actuation and new actuation in order to use spontaneous action of an 
animal. 

[0011] Although an imitation makes an animal copy the same actuation as a 
trainer, that a trainer uses the approach based on an imitation does not have it 
rash. Although it can teach also about continuous actuation or new actuation by 
this approach, this approach is checked only about higher animals, such as the 
primates and a marine mammal. 
[0012] 

[Problem(s) to be Solved by the Invention] By the way, the robot equipment 
made into the configuration which imitated the animal, and the so-called pet 
robot are proposed in recent years. Such robot equipment has a configuration 



similar to the dog bred at ordinary homes, or a cat, and acts autonomously 
according to "it strikes" and the influence of "stroking" from a user (owner), a 
surrounding environment, etc. For example, action of "barking", "sleeping", etc. 
is carried out like an actual animal as autonomous action. 
[0013] If such robot equipment can approach action like an actual animal more, 
the feeling of a living thing of robot equipment will increase further, and a user 
(owner) will sense the much more sense of closeness and satisfaction for robot 
equipment. Thereby, the amusement nature of robot equipment improves. 
[0014] For example, if a user (owner) can teach desired actuation to the robot 
equipment which oneself owns and "can accustom" robot equipment the same 
with receiving an actual animal, a user (owner) will be considered to come to 
sense the much more sense of closeness and satisfaction for robot equipment. 
[0015] However, in order to apply the approach for teaching action to an actual 
animal which was mentioned above to the robot equipment which acts 
autonomously, various problems exist. 

[0016] For example, many modeling mentioned above has been used in order to 
teach a position to robot equipment in the industrial world, but if robot equipment 
acts autonomously and always becomes active, a problem will produce it. That is, 
robot equipment can make a judgment which will sit down if it can sense that the 
trainer is pushing the back and is programmed. However, there is a problem that 



it is not easy to realize the complicated motion of the more than attained to a 
quiescence position by this approach. 

[0017] Moreover, RUARINGU mentioned above is seldom used by robotics. If it 
is programmed so that robot equipment shows interest automatically to a 
predetermined thing, for example, a red thing, a trainer can make robot 
equipment attach to a desired position using this property. However, this also 
has the problem that the use range is restricted. 

[0018] The capture ring mentioned above can create the model of sending that 
signal later, whenever robot equipment takes action of the request relevant to a 
signal (for example, language) using this approach. However, in order to teach 
language like "stability" to robot equipment, a trainer has the problem that it is 
necessary to wait for robot equipment to sit down spontaneously. Moreover, 
there are a problem that there is too many actuation which can name, and a 
problem that it is difficult to teach continuous actuation and new actuation in 
order to use spontaneous action of an animal. 

[0019] the imitation mentioned above - for example, - Y. - in robotics, some 
research consortia have tackled as indicated by Kuniyoshi et al., "Learning by 
watching:Extracting reusable task knowledge from visual observation of human 
performance.", IEEE Transactions on Robotics and Automation, 10(6):799-822, 
1994", etc. Although the continuous actuation and very new actuation can be 



theoretically taught by this approach, since a great quantity of count capacity is 
needed, it is difficult to apply this approach to the existing robot equipment which 
acts autonomously. 

[0020] This invention is proposed in view of such the conventional actual 
condition, and aims at offering the program documentation medium by which the 
action study approach of the robot equipment which can learn complicated and 
continuous action etc., and such robot equipment, the action learning program of 
robot equipment, and the action learning program of the robot equipment 
concerned were recorded. 
[0021] 

[Means for Solving the Problem] In order to attain the purpose mentioned above, 
the robot equipment concerning this invention An action decision means to be 
robot equipment which a right hand side is controlled [ robot ] and makes action 
appear, and to opt for action of 1 based on the appearance inclination of action, 
The above-mentioned right hand side is controlled based on the action for which 
the above-mentioned action decision means opted. An action output means to 
make the action concerned for which it opted appear, and an input means to 
input external information, A primary reinforcer detection means to detect the 
primary reinforcer beforehand set up from the above-mentioned external 
information which the above-mentioned input means inputted, A secondary 



reinforcement factor detection means to detect a secondary reinforcement factor 
from the above-mentioned external information which the above-mentioned 
input means inputted, It has an action storage means to memorize the action for 
which the above-mentioned action decision means opted. The above-mentioned 
action decision means Whenever the above-mentioned secondary 
reinforcement factor is detected, the appearance inclination of action for the 
secondary reinforcement factor concerned to have been detected is raised, and 
the above-mentioned action storage means is characterized by the primary 
reinforcer concerned memorizing the action by which it was detected at least, if 
the above-mentioned primary reinforcer is detected. 

[0022] Here, the action by which the secondary reinforcement factor was 
detected is action until it results in the action by which primary reinforcer was 
detected. 

[0023] Such robot equipment will memorize the action by which the primary 
reinforcer concerned was detected at least, if the appearance inclination of 
action for the secondary reinforcement factor concerned to have been detected 
is raised at the time of study of action and primary reinforcer is detected at it, 
whenever a secondary reinforcement factor is detected. 

[0024] In order to attain the purpose mentioned above, moreover, the action 
study approach of the robot equipment concerning this invention The action 



decision process of being the action study approach of the robot equipment 
which a right hand side is controlled [ robot ] and makes action appearing, and 
opting for action of 1 based on the appearance inclination of action. The 
above-mentioned right hand side is controlled based on the action for which it 
opted at the above-mentioned action decision process. The primary reinforcer 
detection process of detecting the primary reinforcer beforehand set up from the 
above-mentioned external information which an input means to input external 
information as the action output process of making the action concerned for 
which it opted appearing inputted, The secondary reinforcement factor detection 
process of detecting a secondary reinforcement factor from the 
above-mentioned external information which the above-mentioned input means 
inputted, It has the action storage process of memorizing the action for which it 
opted at the above-mentioned action decision process. At the above-mentioned 
action decision process If the appearance inclination of action for the secondary 
reinforcement factor concerned to have been detected is raised and the 
above-mentioned primary reinforcer is detected at the above-mentioned action 
storage process whenever the above-mentioned secondary reinforcement factor 
is detected, it is characterized by memorizing the action by which the primary 
reinforcer concerned was detected at least. 

[0025] Here, the action by which the secondary reinforcement factor was 



detected is action until it results in the action by which primary reinforcer was 
detected. 

[0026] By such action study approach of robot equipment, if the appearance 
inclination of action for the secondary reinforcement factor concerned to have 
been detected is raised at the time of study of action and primary reinforcer is 
detected at it whenever a secondary reinforcement factor is detected, the action 
by which the primary reinforcer concerned was detected at least will be 
memorized. 

[0027] In order to attain the purpose mentioned above, moreover, the action 
learning program of the robot equipment concerning this invention The action 
decision process of being the action learning program of robot equipment which 
a right hand side is controlled [ learning program ] and makes action appearing, 
and opting for action of 1 based on the appearance inclination of action, The 
above-mentioned right hand side is controlled based on the action for which it 
opted at the above-mentioned action decision process. The primary reinforcer 
detection process of detecting the primary reinforcer beforehand set up from the 
above-mentioned external information which an input means to input external 
information as the action output process of making the action concerned for 
which it opted appearing inputted, The secondary reinforcement factor detection 
process of detecting a secondary reinforcement factor from the 



above-mentioned external information which the above-mentioned input means 
inputted, It has the action storage process of memorizing the action for which it 
opted at the above-mentioned action decision process. At the above-mentioned 
action decision process If the appearance inclination of action for the secondary 
reinforcement factor concerned to have been detected is raised and the 
above-mentioned primary reinforcer is detected at the above-mentioned action 
storage process whenever the above-mentioned secondary reinforcement factor 
is detected, it is characterized by memorizing the action by which the primary 
reinforcer concerned was detected at least. 

[0028] Here, the action by which the secondary reinforcement factor was 
detected is action until it results in the action by which primary reinforcer was 
detected. 

[0029] In such an action learning program of robot equipment, if the appearance 
inclination of action for the secondary reinforcement factor concerned to have 
been detected is raised at the time of study of action and primary reinforcer is 
detected at it whenever a secondary reinforcement factor is detected, the action 
by which the primary reinforcer concerned was detected at least will be 
memorized. 

[0030] In order to attain the purpose mentioned above, moreover, the program 
documentation medium concerning this invention It is the program 



documentation medium by which the action learning program of robot equipment 
which a right hand side is controlled [ learning program ] and makes action 
appear was recorded and in which computer control is possible. The action 
decision process that the action learning program of the above-mentioned robot 
equipment opts for action of 1 based on the appearance inclination of action, 
The above-mentioned right hand side is controlled based on the action for which 
it opted at the above-mentioned action decision process. The primary reinforcer 
detection process of detecting the primary reinforcer beforehand set up from the 
above-mentioned external information which an input means to input external 
information as the action output process of making the action concerned for 
which it opted appearing inputted, The secondary reinforcement factor detection 
process of detecting a secondary reinforcement factor from the 
above-mentioned external information which the above-mentioned input means 
inputted, It has the action storage process of memorizing the action for which it 
opted at the above-mentioned action decision process. At the above-mentioned 
action decision process Whenever the above-mentioned secondary 
reinforcement factor is detected, the appearance inclination of action for the 
secondary reinforcement factor concerned to have been detected is raised. At 
the above-mentioned action storage process Detection of the above-mentioned 
primary reinforcer records the action learning program of the robot equipment 



characterized by memorizing the action by which the primary reinforcer 
concerned was detected at least. 

[0031] Here, the action by which the secondary reinforcement factor was 
detected is action until it results in the action by which primary reinforcer was 
detected. 

[0032] In the action learning program of the robot equipment currently recorded 
on such a program documentation medium, if the appearance inclination of 
action for the secondary reinforcement factor concerned to have been detected 
is raised at the time of study of action and primary reinforcer is detected at it 
whenever a secondary reinforcement factor is detected, the action by which the 
primary reinforcer concerned was detected at least will be memorized. 
[0033] 

[Embodiment of the Invention] Hereafter, the gestalt of the concrete operation 
which applied this invention is explained to a detail, referring to a drawing. The 
gestalt of this operation applies this invention to the robot equipment which acts 
autonomously according to a surrounding environment (external factor) or an 
internal condition (inner factor). 

[0034] First, the gestalt of operation explains the configuration of robot 
equipment and explains the application part of this invention in robot equipment 
to a detail after that. 



[0035] (1) As shown in the block diagram 1 of the robot equipment in the gestalt 
of this operation, while considering as the so-called pet robot of the configuration 
which imitated the "dog" and connecting the leg units 3A, 3B, and 3C and 3D 
with front and rear, right and left of the idiosoma unit 2, respectively, the head 
unit 4 and the tail section unit 5 are connected and constituted by the front end 
section and the back end section of the idiosoma unit 2, respectively. 
[0036] As shown in drawing 2 , the control section 16 formed by connecting CPU 
(Central Processing Unit)10, DRAM (Dynamic Random Access Memory)11, a 
flash ROM (Read Only Memory) 12, PC (Personal Computer) card interface 
circuitry 13, and a digital disposal circuit 14 mutually through an internal bus 15 
and the dc-battery 17 as a source of power of this robot equipment are contained 
by the idiosoma unit 2. Moreover, the angular-velocity sensor 18, an 
acceleration sensor 19, etc. for detecting the sense of robot equipment 1 and the 
acceleration of a motion are contained by the idiosoma unit 2. 
[0037] Moreover, the CCD (Charge Coupled Device) camera 20 for picturizing 
an external situation to the head unit 4, The touch sensor 21 for detecting the 
pressure received by "it strokes" and the physical influence of "striking" from a 
user, The distance robot 22 for measuring the distance to the body located 
ahead, LED (Light Emitting Diode) (not shown) equivalent to the microphone 23 
for collecting alien frequencies, the loudspeaker 24 for outputting voice, such as 



a cry, and the "eye" of robot equipment 1 etc. is arranged in the predetermined 
location, respectively. 

[0038] Furthermore, the actuators 251 and 252 for free frequency, ... and 
potentiometers 261 and 262, and ... are arranged in the joining segment of the 
joint part of each leg unit 3A-3D, each joining segment of each leg unit and the 
idiosoma unit 2, the head unit 4, and the idiosoma unit 2, and the list by the 
joining segment of tail 5A of the tail section unit 5, respectively. For example, 
actuators 251 and 252 and ... have the servo motor as a configuration. Leg unit 
3A - 3D are controlled by the drive of a servo motor, and it changes in the target 
posture or actuation. 

[0039] And it connects with the digital disposal circuit 14 of the control section 16 
through the hubs 271-27n corresponding to various sensor lists, such as these 
angular-velocity sensor 18, an acceleration sensor 19, a touch sensor 21, a 
distance robot 22, a microphone 23, a loudspeaker 24 and each potentiometers 
261 and 262, and in LED and each actuators 251 and 252, and 
respectively, and direct continuation of CCD camera 20 and the dc-battery 17 is 
carried out to the digital disposal circuit 14, respectively. 

[0040] A digital disposal circuit 14 incorporates sensor data, and the image data 
and voice data which are supplied from each above-mentioned sensor one by 
one, and carries out sequential storing of these through an internal bus 15 in the 



predetermined location in DRAM11, respectively. Moreover, a digital disposal 
circuit 14 incorporates the dc-battery residue data showing the dc-battery 
residue supplied from a dc-battery 17 with this one by one, and stores this in the 
predetermined location in DRAMII. 

[0041] Thus, each sensor data stored in DRAMII, image data, voice data, and 
dc-battery residue data are used in case CPU 10 performs motion control of this 
robot equipment 1 after this. 

[0042] CPU10 stores direct read-out and this in DRAM1 1 for the control program 
stored in the memory card 28 or flash ROM 12 with which the PC Card slot 
which the idiosoma unit 2 does not illustrate was loaded through the PC card 
interface circuitry 13 at the time of the first stage when the power source of robot 
equipment 1 was switched on in practice. 

[0043] Moreover, as mentioned above, from a digital disposal circuit 14, each 
sensor data by which sequential storing is carried out, image data, voice data, 
and dc-battery residue data are based on DRAM11, and CPU 10 judges the 
situation of self and a perimeter, the existence of the command from a user, and 
influence, etc. after this. 

[0044] Furthermore, CPU 10 can make the head unit 4 able to shake vertically 
and horizontally, can move tail 5A of the tail section unit 5, or makes it act by 
making the required actuators 251 and 252 and ... drive based on the decision 



result concerned to make it walk by making each leg unit 3A-3D drive etc. while it 
opts for the action which continues based on the control program stored in this 
decision result and DRAM1 1 . 

[0045] Moreover, in this case, by generating voice data if needed and giving this 
to a loudspeaker 24 as a sound signal through a digital disposal circuit 14, 
CPU10 makes the voice based on the sound signal concerned output outside, or 
turns on or blinks above-mentioned LED. 

[0046] Thus, in this robot equipment 1. it is made as [ act / according to the 
situation of self and a perimeter the command from a user, and influence / it / 
autonomously ]. 

[0047] (2) the software configuration of a control program - here comes to show 
the software configuration of the above-mentioned control program in robot 
equipment 1 to drawing 3 . In this drawing 3 , the device driver layer 30 is located 
in the lowest layer of this control program, and consists of device driver sets 31 
which consist of two or more device drivers. In this case, each device driver is 
the object allowed to carry out direct access to the hardware used by CCD 
camera 20 ( drawing 2 ) and usual computers, such as a timer, and processes in 
response to interruption from corresponding hardware. 

[0048] Moreover, the ROBOTIKKU server object 32 With the virtual robot 33 
which becomes by the software group which offers the interface for being 



located in the lowest layer of the device driver layer 30, for example, accessing 
hardware, such as above-mentioned various sensors and actuators 251 and 252, 
and ... With the power manager 34 who becomes by the software group which 
manages a switch of a power source etc. It consists of a device driver manager 
35 who becomes by the software group which manages other various device 
drivers, and a dither INDO robot 36 which becomes by the software group which 
manages the device of robot equipment 1. 

[0049] The manager object 37 consists of an object manager 38 and a service 
manager 39. In this case, the object manager 38 is a software group which 
manages starting and termination of each software group contained in the 
ROBOTIKKU server object 32, the middleware layer 40, and the application 
layer 41, and a service manager 39 is a software group which manages 
connection of each object based on the initial entry between each object 
described by the connection file stored in the memory card 28 ( drawing 2 ). 
[0050] The middleware layer 40 is located in the upper layer of the ROBOTIKKU 
server object 32, and consists of software groups which offer the fundamental 
function of these robot equipments 1, such as an image processing and speech 
processing. Moreover, the application layer 41 is located in the upper layer of the 
middleware layer 40, and consists of software groups for opting for action of 
robot equipment 1 based on the processing result processed by each software 



group which constitutes the middleware layer 40 concerned. 

[0051] In addition, the concrete software configuration of the middleware layer 

40 and the application layer 41 is shown in drawing 4 . 

[0052] As shown in drawing 4 , in the middleware layer 40 For noise detection, 
The object for temperature detection, the object for brightness detection, the 
object for scale recognition, the object for distance detection, for posture 
detection, The recognition system 60 which has the input semantics converter 
module 59 etc. in the object for touch sensors, the object for motion detection, 
and each signal conditioning module 50 for color recognition - 58 lists, It consists 
of output systems 69 which have each signal conditioning modules 61-67 for the 
object for posture management, the object for tracking, the object for motion 
playback, the object for a walk, the object for a fall return, the object for LED 
lighting, and sound playback etc. in output semantics converter module 68 list. 
[0053] Each signal conditioning modules 50-58 of the recognition system 60 
incorporate the data with which it corresponds of each sensor data read from 
DRAM1 1 ( drawing 2 ) by the virtual robot 33 of the ROBOTIKKU server object 
32, or image data and voice data, perform predetermined processing based on 
the data concerned, and give a processing result to the input semantics 
converter module 59. Here, the virtual robot 33 is constituted by the 
predetermined protocol as a part which carries out transfer or conversion of a 



signal. 

[0054] The input semantics converter module 59 It is based on the processing 
result given from these signal conditioning modules 50-58. "The fall was 
detected", [ "it is "noisy", hot / "hot" /, and bright", "the ball having been detected", 
and ] The self of "it was stroked", "it having been struck", "the scale of C-E-G 
having been heard", "the body which moves having been detected", "having 
detected the obstruction", etc. and a surrounding situation, the command from a 
user, and influence are recognized, and a recognition result is outputted to the 
application layer 41 ( drawing 3 ). 

[0055] The application layer 41 consists of five modules, the behavioral model 
library 70, the action change module 71, the study module 72, the feeling model 
73, and the instinct model 74, as shown in drawing 5 . 

[0056] As shown in drawing 6 , "when a dc-battery residue decreases, and 
carrying out a fall return, and expressing [ avoiding an obstruction, and ] feeling", 
the behavioral models 701 -70n which became independent, respectively "are 
formed [ the behavioral model library 70 is made to correspond to the condition 
item of the shoes "at the time of detecting a ball" etc. chosen beforehand, 
respectively, and ]". [ "being formed in it" ] [ "being formed" ] 
[0057] And these behavioral models 701 -70n The time of a recognition result 
being given from the input semantics converter module 59, respectively, The 



parameter value of the corresponding emotion currently held like the 
after-mentioned at the feeling model 73 if needed when fixed time amount has 
passed, after the last recognition result is given, It opts for the action which 
continues while referring to the parameter value of the corresponding desire 
currently held at the instinct model 74, respectively, and a decision result is 
outputted to the action change module 71. 

[0058] In the case of the gestalt of this operation, in addition, each behavioral 
models 701-70n As the technique of opting for the next action As opposed to the 
arc ARC1 - ARCn which connect between each node NODEO - NODEn(s) for to 
other nodes NODEO of which - NODEn it changes from one node (condition) 
NODEO as shown in drawing 7 - NODEn The algorithm called the finite 
stochastic automaton determined probable based on the transition probability 
P1-Pn by which a ******** setup was carried out is used. 

[0059] Concretely, each behavioral models 701-70n are made to correspond to 
the node NODEO which forms the self behavioral models 701 -70n. respectively - 
NODEn. respectively, and have the state transition table 80 as shown in drawing 
8 for every these node NODEO - NODEn. 

[0060] In this state transition table 80, the input event (recognition result) made 
into transition conditions in that node is listed by the train of an "input event 
name" at a priority, and the further conditions about that transition condition are 



described by the corresponding line in the train of a "data name" and the "data 
range." 

[0061] therefore, in the node NODE100 expressed in the state transition table 80 
of drawing 8 When the recognition result of "detecting a ball (BALL)" is given The 
range of "magnitude (SIZE)" of the ball given with the recognition result 
concerned is "0 to 1000", When the recognition result of "detecting an 
obstruction (OBSTACLE)" is given, they have been conditions for that the range 
of "the distance (DISTANCE)" to the obstruction done with the recognition result 
concerned is "0 to 100" to change to other nodes. 

[0062] Moreover, in this node NODE100, when there is no input of a recognition 
result, it also sets. The inside of each emotion held at the feeling model 73 and 
the instinct model 74 which 701 -70n of behavioral models refers to periodically, 
respectively, and the parameter value of each desire, it was held at the feeling 
model 73 "-- glad (JOY) - " - "- surprised (SURPRISE) " or - "- feeling 
sad (SADNESS) - " - when the range of one of parameter value is "50 to 100". 
it can change to other nodes. 

[0063] moreover - a state transition table 80 - "— others, while the node name 
which can change from the node NODEO - NODEn in the line of the "transition 
place node" in the column of transition probability" to a node is listed It is 
described by the part where it corresponds in the column of transition probability" 



to a node, respectively, the transition probability to each of other node NODEO 
which can change when ail the conditions described by the "input event name", 
the "data value", and the train of "the range of data" are met - NODEn "- 
others - the action which should be outputted in case it changes to the node 
NODEO - NODEn -- "- others - it is described by the line of "output action" in the 
column of transition probability" to a node, in addition - "- others the sum of 
the probability of each line in the column of transition probability" to a node is 
100 [%]. 

[0064] therefore, in the node NODE100 expressed in the state transition table 80 
of drawing 8 for example, when the recognition result that it carries out 
"detecting a ball (BALL)", and the range of "magnitude (SIZE)" of the ball is "0 to 
1000" is given It can change to "a node NODE 120 (node 120)" by the probability 
of "30 [%r, and action of "ACTION 1" will be then outputted. 
[0065] When they are constituted as a lot of nodes NODEO described as such 
[ respectively ] a state transition table 80 - NODEn(s) are connected, and a 
recognition result is given from the input semantics converter module 59, each 
behavioral models 701 -70n opt for the next action probable using the state 
transition table 80 of the node NODEO - NODEn, and are made as [ output / to 
the action change module 71 / a decision result ]. 

[0066] The action change module 71 shown in drawing 5 chooses from each 



behavioral models 701 -70n of the behavioral model library 70 the action 
outputted from the high behavioral models 701-70n of the priority beforehand 
defined among the actions outputted, respectively, and sends out the command 
(this is hereafter called action command.) of the purport which should perform 
the action concerned to the output semantics converter module 68 of the 
middleware layer 40. In addition, in the gestalt of this operation, priority is set up 
highly about 701 -70n of behavioral models written by the bottom in drawing 6 . 
[0067] Moreover, the action change module 71 notifies that the action was 
completed based on the completion information of action given from the output 
semantics converter module 68 after the completion of action to the study 
module 72, the feeling model 73, and the instinct model 74. 
[0068] On the other hand, the recognition result of the instruction received as 
influence from a user, such as the study module 72 "was struck" among the 
recognition results given from the input semantics converter module 59 and "it 
having been stroked", is inputted. 

[0069] And based on the notice from this recognition result and the action 
change module 71, the study module 72 reduces the manifestation probability of 
that action, when "struck" (scolded), and when "stroked" (praised), it changes 
behavioral models [ in the behavioral model library 70 / 701 -70n ] corresponding 
transition probability so that the manifestation probability of that action may be 



raised. 

[00701 on the other hand, the feeling model 73 - glad (joy) - " - feeling 
sad (sadness) - " - getting angry (anger) - " -- surprised (surprise) -- " - 
"dislike (disgust)" - and - "- afraid (fear) - " - the parameter with which the 
strength of the emotion is expressed for every emotion is held about a total of six 
emotions. And the feeling model 73 updates the parameter value of each 
[ these ] emotion periodically based on the notice from the specific recognition 
result to which it is given from the input semantics converter module 59, 
respectively, such as "it having been struck" and "it having been stroked", and 
elapsed time and the action change module 71 etc. 

[0071] The recognition result to which the feeling model 73 is specifically given 
from the input semantics converter module 59, The variation of the action and its 
emotion when being computed by predetermined operation expression based on 
the elapsed time after updating last time etc. of the robot equipment 1 at that 
time deltaE [t]. The multiplier which expresses the sensibility of E [t] and its 
emotion for the parameter value of the current emotion is set to ke. (1) By the 
formula, as parameter value [ of the emotion in the following period ] E [t+1] is 
computed and this is replaced with parameter value [ of the current emotion ] E 
[t], the parameter value of the emotion is updated. Moreover, the feeling model 
73 updates the parameter value of all emotions like this. 



[0072] 
[Equation 1] 

E[t+l]=E[t]+kc xAE[t] (1) 

[0073] In addition, it is decided beforehand how many effects the notice from 
each recognition result or the output semantics converter module 68 has on 
amount of fluctuation delta[ of the parameter value of each emotion ] E [t]. For 
example, the recognition result of "having been struck" has big effect on amount 
of fluctuation delta[ of the parameter value of the emotion of the "resentment" ] E 
[t], and the recognition result of "having been stroked" has big effect on amount 
of fluctuation delta[ of the parameter value of the emotion of "joy" ] E [t]. 
[0074] Here, the notice from the output semantics converter module 68 is the 
so-called feedback information (the completion information of action) of action, 
and is the information on the appearance result of action, and the feeling model 
73 changes feeling also using such information, this — for example, the feeling 
level of the resentment falls by action of "barking" - like - they are things. In 
addition, the notice from the output semantics converter module 68 is inputted 
also into the study module 72 mentioned above, and the study module 72 
changes behavioral models [ 701 -70n ] corresponding transition probability 
based on the notice. 



[0075] In addition, feedback of an action result may be made with the output 
(action to which feeling was added) of the action change module 71. 
[0076] on the other hand, "movement avarice (exercize)", "love avarice 
(affection)", "appetite (appetite)", and "the curiosity (curiosity) of the instinct 
model 74" are mutually-independent the parameter with which the strength of 
the desire is expressed for these the desires of every is held about four desires 
the bottom. And the instinct model 74 updates the parameter value of these 
desires periodically based on the recognition result to which it is given from the 
input semantics converter module 59, respectively, the notice from elapsed time 
and the action change module 71, etc. 

[0077] The instinct model 74 specifically about "movement avarice", "love 
avarice", and "curiosity" The amount of fluctuation of that the desire of the when 
being computed by predetermined operation expression based on the notice 
from a recognition result, elapsed time, and the output semantics converter 
module 68 etc. delta! [k]. The multiplier which expresses the sensibility of I [k] 
and its desire for the parameter value of the current desire is set to ki. As 
parameter value [ of that desire in the following period ] I [k+1] is computed using 
(2) types with a predetermined period and this result of an operation is replaced 
with parameter value [ of that current desire ] I [k], the parameter value of that 
desire is updated. Moreover, the instinct model 74 updates the parameter value 



of each desire except "appetite" like tiiis. 

[0078] 

[Equation 2] 

I[k+l]=I[k]-fki xAl[k] ' • • 

[0079] In addition, it is decided beforehand how many effects the notice from a 
recognition result and the output semantics converter module 68 etc. has on 
amount of fluctuation delta[ of the parameter value of each desire ] I [k], for 
example, it has effect to amount of fluctuation delta[ of the parameter value of 
the "fatigue" ] I [k] with the big notice from the output semantics converter 
module 68. 

[0080] In addition, in the gestalt of this operation, it is regulated so that each 
emotion and the parameter value of each desire (instinct) may be changed in the 
range from 0 to 100, respectively, and the value of multipliers ke and ki is also 
set up according to the individual for each [ an emotion and ] the desire of every. 
[0081] On the other hand, abstract action commands, such as it being [ which is 
given from the action change module 71 of the application layer 41 as mentioned 
above ] "advance", "it being glad", the output semantics converter module 68 of 
the middleware layer 40 "cries", as shown in drawing 4 , or "tracking (a ball is 
pursued)", are given to the signal conditioning modules 61-67 with which the 



output system 69 corresponds. 

[0082] And these signal conditioning modules 61-67 The servo command value 
which should be given to the actuators 251 -25n ( drawing 2 ) in order to perform 
this action based on the action command concerned, if an action command is 
given, The drive data given to the voice data of the sound outputted from a 
loudspeaker 24 ( drawing 2 ) and/or LED of a "eye" are generated. Sequential 
sending out of these data is carried out at the actuators 251 -25n which 
correspond through the virtual robot 33 and digital disposal circuit 14 ( drawing 
2 ) of the ROBOTIKKU server object 32 one by one, a loudspeaker 24, or LED. 
[0083] Thus, in robot equipment 1, it is made based on the control program as 
[ perform / the situation of self (interior) and a perimeter (exterior), the directions 
from a user, and autonomous action according to influence ]. 
[0084] (3) Use the approach called shaping (Shaping) to robot equipment 1 as 
an approach of teaching action with the gestalt of the action instruction approach 
book operation used with the gestalt of this operation. By this approach, in order 
to form action, action is divided into the continuous small response which can be 
attained, and, finally it leads to desired action. That is, this approach is gradually 
guided towards suitable action. Although various techniques can be used for 
performing each phase, with the gestalt of this operation, the approach called 
clicker training which has spread as the training approach of an animal is used. 



Hereafter, this clicker training is explained. 

[0085] Clicker training is based on the theory of the operant conditioning of 
B.F.Skinner indicated by "B. F.Skinner, "The Behavior of Organisms., Appleton 
Century Crofs, New York, NY., 1938." Karen Pryor which is a behaviorist 
psychology person's Gary Wilkes and the trainer of a dolphin spread the training 
approach of a dog based on this clicker training through the 1980s together. By 
clicker training, the clicker which is a metal small toy is used instead of the 
whistle currently conventionally used for training of a dolphin. 
[0086] A push on a clicker emits a short sharp sound. Any semantics cannot be 
found for an animal only to this sound. However, a trainer relates this sound with 
primary reinforcer. An animal senses primary reinforcer instinctive as 
remuneration like food or a toy. The sound of a clicker serves as a secondary 
reinforcement factor (conditioned reinforcer) by relating primary reinforcer and 
many times and carrying out. This 2nd reinforcer carries out the role of the 
positive hint that remuneration can be got soon, for an animal. The clicker itself 
is used in order to lead not remuneration but an animal to desired action for an 
animal. Moreover, an animal can be told about which action should be 
strengthened with this clicker. A trainer gives primary reinforcer, only when an 
animal carries out desired action. This is the signal showing termination of a 
series of induction processes led to desired action. 



[0087] Clicker training consists of four steps shown below at least. 
[0088] The 1st step: Carry out the charge up of the clicker. In this step, an animal 
has prizes (food etc.) and chain attachment ****** in the sound of a clicker. In it, a 
clicker is sounded, and continuously, it repeats giving remuneration continuously 
about many times, for example, 20 to 50 times, until an animal comes to be 
clearly excited to the sound of a clicker. 

[0089] The 2nd step: Catch action. Next, it guides so that actuation of a request 
of an animal may be taken. For example, when it is thought that I want an animal 
for a trainer to draw and turn around a circle in the clockwise direction, and the 
head of an animal moves to the right a little, it begins from sounding a clicker. If 
an animal comes to move the head repeatedly, a clicker will be sounded only 
when beginning to move the body to the right this time. Remuneration will be 
given, if the criteria with which a clicker is sounded gradually are raised and the 
body rotates one time completely. 

[0090] The 3rd step: Give a command word. If an animal learns desired action, a 
command word will be said only once. The timing which says a command word 
is just before immediately after an animal performs the action. 
[0091] The 4th step: Try action. Next, it is necessary to try and refine the learned 
action. Only when action with an exact animal is taken, a command word is said 
and remuneration is given. 



[0092] As explained above, clicker training is for leading an aninnal to desired 
action. Furthermore, it can be used that an animal performs this clicker training 
spontaneously in order to make the rare action which is not rash learn, or in 
order to make a series of continuous actions learn. 

[0093] Below, how the robot equipment 1 mentioned above by using the 
principle of this clicker training learns action is explained. 

[0094] (4) Explain the important section to which this invention was applied in the 
application above-mentioned robot equipment 1 to robot equipment. Robot 
equipment 1 is related with a piicture signal or a sound signal (acoustic signal), 
action is learned, and after study is constituted so that the picture signal and 
sound signal which were associated may cause action. Although the following 
explanation mainly explains the example which relates voice with action as study, 
it cannot be overemphasized that an image can be related with action, either. 
Specifically, robot equipment 1 has the following configurations for 
implementation of this invention. 

[0095] Robot equipment 1 is equipped with the input section 101, the primary 
reinforcer detecting element 102, the secondary reinforcement factor associative 
section 103, the action decision section 104, the action storage section 105, the 
action output section 106, the speech recognition section 107, and the 
associative storage section 108 as shown in drawing 9 . 



[0096] Here, the input section 101 is the microphone 23 and touch sensor 21 
which are shown in drawing 2 . The input section 101 supplies a sound signal or 
a sensor signal to the primary reinforcer detecting element 102 and the 
secondary reinforcement factor associative section 103. Moreover, the input 
section 101 supplies a sound signal to the speech recognition section 107. 
[0097] The primary reinforcer detecting element 102 detects primary reinforcer 
from the sound signal supplied from the input section 101, or a sensor signal. 
Primary reinforcer is set up so that robot equipment 1 may sense as 
remuneration beforehand, and detection of the pressure (what depends the 
head on striking lightly) to the head of robot equipment 1, the sound signal with 
which it loves and **** is expressed are mentioned. Below, the sound signal of 
"bravo" is used as primary reinforcer. The primary reinforcer detecting element 
102 will supply primary strengthening signals to the action decision section 104 
and the action storage section 105, if primary reinforcer is detected. 
[0098] The secondary reinforcement factor associative section 103 reminds of it 
and detects a secondary reinforcement factor from the sound signal supplied 
from the input section 101, or a sensor signal. The secondary reinforcement 
factor may be beforehand set as robot equipment 1 like primary reinforcer, and a 
trainer relates it with primary reinforcer, and you may make it set it up. When a 
trainer sets up a secondary reinforcement factor, robot equipment 1 is made into 



secondary reinforcement factor setting mode, and it sets up by [ as being shown 
in drawing 10 ]. 

[0099] As shown in drawing 10 , in case a secondary reinforcement factor is set 
up, in step SI, the stimulus used as a secondary reinforcement factor is given 
first. Even if this stimulus is a contact stimulus and it is an acoustic stimulus, it is 
not cared about. 

[0100] Then, in step S2, the stimulus which gave primary reinforcer and was 
given at step SI in step S3 is related with primary reinforcer, and it memorizes 
as a secondary reinforcement factor. 

[0101] In addition, if it detects, for example that primary reinforcer was given 
after the stimulus which is not limited to this example, for example, has robot 
equipment 1 30 times or more many times, the setting technique of a secondary 
reinforcement factor may be beforehand set up so that that stimulus may be 
memorized as a secondary reinforcement factor. Moreover, modification of the 
set-up secondary reinforcement factor can also be enabled. A secondary 
reinforcement factor can be changed by giving a stimulus of arbitration, giving in 
this case, for example, a secondary reinforcement factor change command. 
[0102] In addition, this secondary reinforcement factor can be given quickly, and 
it must be easy to detect it so that it can function as an indicator which excelled 
for guiding robot equipment 1 to suitable action. Below, the sound signal of 



"good" is used as a secondary reinforcement factor. 

[0103] The secondary reinforcement factor associative section 103 will supply a 
secondary reinforcement signal to the action decision section 104 and the action 
storage section 105, if a secondary reinforcement factor is detected. 
[0104] The action decision section 104 opts for the next action probable based 
on the transition probability to other nodes [ node / (condition) / a certain ], and 
supplies a decision result to the action output section 106. Moreover, the action 
decision section 104 mai<es the action storage section 105 supply and memorize 
the hysteresis of the action for which it opted. Furthermore, based on primary 
strengthening signals and secondary reinforcement signal which were supplied 
from the primary reinforcer detecting element 102 and the secondary 
reinforcement factor associative section 103, the action decision section 104 
updates transition probability, or it updates transition probability so that it may be 
easy to change to the node (condition) and may become it from the last node 
(condition), so that the selection probability of a certain node (condition) may 
become high, so that it may mention later. 

[0105] The action storage section 105 memorizes the hysteresis of the action for 
which it opted in the action decision section 104. Moreover, to the action storage 
section 104, primary strengthening signals and secondary reinforcement signal 
from the primary reinforcer detecting element 102 and the secondary 



reinforcement factor associative section 103 are supplied, and the action to 
which the secondary reinforcement signal was given after becoming action 
learning mode before primary strengthening signals were supplied is memorized 
as a series of actions in it. 

[0106] The action output section 106 is sent out to the output semantics 
converter module 68 which shows the action command mentioned above to 
drawing 4 based on the decision result of the action supplied from the action 
decision section 105. 

[0107] The speech recognition section 107 carries out speech processing of the 
sound signal inputted from the input section 101, and recognizes it as 
predetermined language. HMM (HiddenMarkov Model) is used for the speech 
recognition section 107, it is constituted, and, specifically, recognizes the 
inputted voice as a phoneme sequence by two or more recognition classes 
depended on HMM. 

[0108] The associative storage section 108 associates and carries out 
associative storage of the command showing a series of actions to which the 
secondary reinforcement signal was given after becoming action learning mode 
before primary strengthening signals were supplied, and the action concerned 
recognized in the speech recognition section 107. A series of actions by which 
associative storage was carried out by relating with a command are memorized 



by the action storage section 104. 

[0109] A trainer can guide robot equipment 1 to desired action using a 
secondary reinforcement factor because robot equipment 1 has the above 
configurations. That is, a trainer will give a secondary reinforcement factor, if 
robot equipment 1 approaches desired action, and he is taken as the indicator 
for approaching the action. If robot equipment 1 finally acts a request, a trainer 
will give primary reinforcer and robot equipment will memorize a series of actions 
to which the secondary reinforcement factor till then was given. And if a trainer 
utters a command, robot equipment will output a series of memorized actuation 
in order. In addition, it is also possible to narrow them down, as a series of 
memorized actions are mentioned later. 

[0110] A series of procedures of this action study approach are shown in 
drawing 11 . As shown in drawing 1 1 , in step S10. it is first set as action learning 
mode. 

[0111] Next, in step S11, the transition probability from a node to a node is 
initialized, and in step SI 2, robot equipment 1 opts for action according to 
current transition probability, and outputs the action. 

[0112] At step S13, it is distinguished whether primary reinforcer was given by 
action outputted at step S12. When primary reinforcer is given, it progresses to 
step S17. In step SI 3, when primary reinforcer is not given, it progresses to step 



S14. 

[0113] At step S14, it is distinguished whether the secondary reinforcement 
factor was given by action outputted at step SI 2. When the secondary 
reinforcement factor is not given, it supposes that it is not action for approaching 
desired action, it returns to step SI 2, and opts for the next action again. In step 
S14, when a secondary reinforcement factor is given, it progresses to step SI 5. 
[0114] At step SI 5, it memorizes as action which results in action of a request of 
the action outputted at step SI 2, and progresses to step SI 6. 
[0115] At step S16, transition probability is updated and it returns to step SI 2 so 
that the selection probability of action for the secondary reinforcement signal to 
have been given may become high. 

[0116] At step S17, the command for relating with the signal with which primary 
strengthening signals were given is uttered. 

[0117] At step S18, the action to which the secondary reinforcement signal was 
given by the time it resulted in the action to which primary strengthening signals 
were given, and its action is related with the command uttered at step SI 7 as a 
series of actions, and it memorizes and ends. 

[0118] By the action study approach in the gestalt of this operation, robot 
equipment 1 is guided to desired action by the above procedures, and the action 
is learned. 



[0119] In addition, although the command related with desired action was 
uttered in the flow chart of drawing 11 after initializing transition probability, you 
may make it utter the command related with the action immediately after it is not 
limited to this and robot equipment 1 acts a request, 

[0120] Moreover, although the flow chart of drawing 11 explained as what 
updates transition probability so that the selection probability of action for the 
secondary reinforcement signal to have been given might become high in step 
SI 6, it is not limited to this and you may make it update transition probability so 
that it may be easy to change to the action to which the secondary reinforcement 
signal was given from the last action and may become it. 

[0121] By the way, although a trainer can guide robot equipment 1 to desired 
actuation using a secondary reinforcement factor, he needs to define the 
"phase" of action of robot equipment 1 as this first. Namely, after robot 
equipment 1 takes a certain action, it and the change to "similar" action must be 
possible for it. As an approach for defining the phase of existing action, two 
examples are given to below. 

[0122] The 1st approach explains each action using 1 set of properties according 
to which it is classified into an explanatory property and an intention-property 
first. 

[0123] An explanatory property is for example, an initiation posture (it stands. 



sits down and lies), and whether a robot's head, a foot, other parts, or a sound is 
emitted actually relates to this. 

[0124] An intention-property is the purpose which makes action take, for 
example, the thing for moving, or expresses whether it is for obtaining whether it 
is and the attention for holding something. Each action can be seen as a point in 
the space defined by these dimensions. All actions can be format-ized according 
to this property, and it can guide to the action "was similar" from a certain action 
by defining the "distance" during two actions. 

[0125] Although all actions can be considerably expressed by this approach in a 
short time once a property is chosen, it is hard to predict the transition during 
action. 

[0126] Next, the 2nd approach creates the graph which shows the transition 
during each action. According to this approach, transition of the action which 
robot equipment 1 takes can be grasped more effectively. It is possible like the 
1st approach to combine the objective similar point during action with a certain 
criteria relevant to "intention nature." Furthermore, general action (it stands 
[ sitting down, ]) and rare action (the gymnastics which dance a special dance 
are carried out) can be distinguished more clearly. A part of phase defined as 
drawing 12 in the graph former rhythm is shown. Moreover, a part of condition of 
having taken the action to which robot equipment 1 was mentioned to drawing 



12 is shown in drawing 13 . 

[0127] A trainer explains concretely the case where the language for the 
excavation action whose robot equipment 1 is the rarely taken special action is 
taught, using drawing 12 . In this action, as shown in drawing 13 (G), a forward 
left foot is used for robot equipment 1 sitting down and scratching the ground. 
Moreover, the head is looking at the foot and follows the motion. This action 
corresponds to the condition of the node [DIG] of drawing 12 . 
[0128] First, robot equipment 1 presupposes that it has started in the condition of 
a node [STAND] as shown in drawing 13 (A). As first shown in drawing 13 (B), 
robot equipment 1 begins a walk, namely, changes in the condition of a node 
[WALK] from the condition of a node [STAND]. Since the transition to this node 
[WALK] does not approach the node [DIG] which is in a desired condition, a 
trainer gives no reinforcer. Then, robot equipment 1 starts another action from 
the condition of a node [STAND]. In addition, you may make it return to the 
condition of a node [STAND] automatically, and may make it return according to 
transition probability in this case. 

[0129] Next, robot equipment 1 sits down namely, changes to a node [SIT], as 
shown in drawing 13 (D). Since the transition to this node [SIT] is a thing 
approaching the node [DIG] which is in a desired condition, a trainer says "good" 
and gives a secondary reinforcement factor to robot equipment 1. Transition 



probability is updated by this secondary reinforcement factor being given, and 
robot equipment 1 tends to take the action (node [SIT]) which is the action to 
which the secondary reinforcement factor was given and which sits down, and 
the associated action. 

[0130] Then, as shown in drawing 13 (E), it is begun to push robot equipment 1 
by two nose gear. That is, it changes to a node [PUSH]. However, since the 
transition to this node [PUSH] does not approach the node [DIG] which is in a 
desired condition, a trainer gives no reinforcer. Then, robot equipment 1 starts 
another action from the condition of a node [SIT]. You may make it return to the 
condition of a node [SIT] automatically, and may make it return according to 
transition probability also in this case. 

[0131] Then, robot equipment 1 changes to a node [HELLO], as shown in 
drawing 13 (F), raises a forward left foot and operates a "halo." Since the 
forward left foot which uses this action for excavation is related, a trainer tells 
"good" to robot equipment 1 and gives a secondary reinforcement factor. 
[0132] If robot equipment 1 changes to a node [DIG] and starts excavation action 
through the exchange with the above trainers and robot equipment 1, a trainer 
will say "bravo" and will give primary reinforcer to robot equipment 1. As for this 
primary reinforcer, robot equipment 1 means having acted the request. Robot 
equipment 1 memorizes a series of actions called [SIT]-[HELLO]-[DIG] the 



above result. 

[0133] In addition, even if it uses not only the path of [SIT]-[HELLO]-[DIG] 
mentioned above but another path, robot equipment 1 can be guided to desired 
action. For example, a trainer can move crus sinistrum diaphragmatis to robot 
equipment 1 through the action to kick as shown in drawing 13 (C), i.e.. a node, 
[KICK], and he can guide so that it may sit on a degree. 

[0134] If desired action is performed and primary reinforcer is given, a trainer will 
give the command word immediately related with action to robot equipment 1. In 
order to check that surely robot equipment 1 has understood the command word 
in that case, it is desirable to adopt a certain feedback. As feedback, robot 
equipment 1 imitates the command through the loudspeaker 24 shown in 
drawing 2 , for example, and you may make it require a check, and the purport 
that he understood something by blinking LED which is a "eye" etc. is shown, 
and you may make it wait to repeat a command again repeatedly. When robot 
equipment 1 cannot detect the command again given by the trainer, you may 
make it the special action (for example, for the head to be shaken) for doing the 
thing same once again express that he was not able to understand a command 
to a trainer. 

[0135] Robot equipment 1 performs a series of actions called 
[SIT]-[HELLO]-[DIG] of a memorized single string acted for example, mentioned 



above, when this command word and a series of actions mentioned above are 
associated and memorized and a command word is given again. 
[0136] When a trainer wants to associate the excavation action of a series of 
actions, and a command word here, it is necessary to narrow down the action of 
a series of. For this reason, if a command word is given, robot equipment 1 can 
repeat a series of memorized actuation spontaneously. When primary reinforcer 
is given after robot equipment 1 performed a series of actuation again, it is 
judged that robot equipment 1 is connected with action of a single string 
[ command / the ]. When that is not right, although it originates in a series of 
actions performed before, rather than it, robot equipment 1 makes little the new 
actions of the number of phases of a series of, and performs the new action of a 
series of. Thus, finally the command word can judge that robot equipment 1 
relates only to excavation action. 

[0137] As explained above, the robot equipment 1 in the gestalt of this operation 
can learn the action efficiently by teaching action like the time of a trainer 
teaching an animal and a man action. Moreover, since it is guided to desired 
action by using a secondary reinforcement factor as an indicator in that case, the 
count of the trial-and-error when learning complicated actuation and the action 
which is not performed rarely can be reduced. Furthermore, when a secondary 
reinforcement factor is given, by updating transition probability, the probability 



which changes to desired action becomes high, and becomes possible 
[ reaching early by desired action ]. 

[0138] By such a function being included in robot equipment 1, communication 
with natural user (owner) and robot equipment 1 can be aimed at. In addition, 
this control program is stored in the memory card 28 or flash ROM 12 of drawing 
2 . 

[0139] In addition, as for this invention, it is needless to say for various 
modification to be possible in the range which is not limited only to the gestalt of 
operation mentioned above and does not deviate from the summary of this 
invention. 

[0140] For example, although speech information was used as a command 
related with action with the gestalt of operation mentioned above, you may be 
the image information through the contact information which is not limited to this 
and minded the touch sensor, or CCD. 

[0141] Moreover, although clicker training was used with the gestalt of operation 
mentioned above when teaching action to robot equipment, it is not limited to 
this and can use for other purposes. That is, when teaching an objective 
identifier, robot equipment is guided to the object by giving a secondary 
reinforcement factor, in case robot equipment approaches an object. Thereby, 
the difficult problem of joint cautions is mitigable. 



[0142] Moreover, you may make it combine clicker training and the lure ring 
mentioned above. It becomes unnecessary thereby, to wait to perform action of 
a certain phase spontaneously. 
[0143] 

[Effect of the Invention] The robot equipment applied to this invention as 
explained to the detail above An action decision means to be robot equipment 
which a right hand side is controlled [ robot ] and makes action appear, and to 
opt for action of 1 based on the appearance inclination of action, The 
above-mentioned right hand side is controlled based on the action for which the 
above-mentioned action decision means opted. An action output means to make 
the action concerned for which it opted appear, and an input means to input 
external information, A primary reinforcer detection means to detect the primary 
reinforcer beforehand set up from the above-mentioned external information 
which the above-mentioned input means inputted, A secondary reinforcement 
factor detection means to detect a secondary reinforcement factor from the 
above-mentioned external information which the above-mentioned input means 
inputted, It has an action storage means to memorize the action for which the 
above-mentioned action decision means opted. The above-mentioned action 
decision means Whenever the above-mentioned secondary reinforcement factor 
is detected, the appearance inclination of action for the secondary reinforcement 



factor concerned to have been detected is raised, and the above-mentioned 
action storage means is characterized by the primary reinforcer concerned 
memorizing the action by which it was detected at least, if the above-mentioned 
primary reinforcer is detected. 

[0144] Here, the action by which the secondary reinforcement factor was 
detected is action until it results in the action by which primary reinforcer was 
detected. 

[0145] Such robot equipment will memorize the action by which the primary 
reinforcer concerned was detected at least, if the appearance inclination of 
action for the secondary reinforcement factor concerned to have been detected 
is raised at the time of study of action and primary reinforcer is detected at it, 
whenever a secondary reinforcement factor is detected. 

[0146] The robot equipment with which this invention was applied can be given 
to the action which is going to learn a secondary reinforcement factor as an 
indicator, and can learn action efficiently. 

[0147] In order to attain the purpose mentioned above, moreover, the action 
study approach of the robot equipment concerning this invention The action 
decision process of being the action study approach of the robot equipment 
which a right hand side is controlled [ robot ] and makes action appearing, and 
opting for action of 1 based on the appearance inclination of action, The 



above-mentioned right hand side is controlled based on the action for which it 
opted at the above-mentioned action decision process. The primary reinforcer 
detection process of detecting the primary reinforcer beforehand set up from the 
above-mentioned external information which an input means to input external 
information as the action output process of making the action concerned for 
which it opted appearing inputted, The secondary reinforcement factor detection 
process of detecting a secondary reinforcement factor from the 
above-mentioned external information which the above-mentioned input means 
inputted, It has the action storage process of memorizing the action for which it 
opted at the above-mentioned action decision process. At the above-mentioned 
action decision process If the appearance inclination of action for the secondary 
reinforcement factor concerned to have been detected is raised and the 
above-mentioned primary reinforcer is detected at the above-mentioned action 
storage process whenever the above-mentioned secondary reinforcement factor 
is detected, it is characterized by memorizing the action by which the primary 
reinforcer concerned was detected at least. 

[0148] Here, the action by which the secondary reinforcement factor was 
detected is action until it results in the action by which primary reinforcer was 
detected. 

[0149] By such action study approach of robot equipment, if the appearance 



inclination of action for the secondary reinforcement factor concerned to have 
been detected is raised at the time of study of action and primary reinforcer is 
detected at it whenever a secondary reinforcement factor is detected, the action 
by which the primary reinforcer concerned was detected at least will be 
memorized. 

[0150] The robot equipment with which this invention was applied can be given 
to the action which is going to learn a secondary reinforcement factor as an 
indicator, and can learn action efficiently. 

[0151] In order to attain the purpose mentioned above, moreover, the action 
learning program of the robot equipment concerning this invention The action 
decision process of being the action learning program of robot equipment which 
a right hand side is controlled [ learning program ] and makes action appearing, 
and opting for action of 1 based on the appearance inclination of action. The 
above-mentioned right hand side is controlled based on the action for which it 
opted at the above-mentioned action decision process. The primary reinforcer 
detection process of detecting the primary reinforcer beforehand set up from the 
above-mentioned external information which an input means to input external 
information as the action output process of making the action concerned for 
which it opted appearing inputted, The secondary reinforcement factor detection 
process of detecting a secondary reinforcement factor from the 



above-mentioned external information which the above-mentioned input means 
inputted, It has the action storage process of memorizing the action for which it 
opted at the above-mentioned action decision process. At the above-mentioned 
action decision process If the appearance inclination of action for the secondary 
reinforcement factor concerned to have been detected is raised and the 
above-mentioned primary reinforcer is detected at the above-mentioned action 
storage process whenever the above-mentioned secondary reinforcement factor 
is detected, it is characterized by memorizing the action by which the primary 
reinforcer concerned was detected at least. 

[0152] Here, the action by which the secondary reinforcement factor was 
detected is action until it results in the action by which primary reinforcer was 
detected. 

[0153] In such an action learning program of robot equipment, if the appearance 
inclination of action for the secondary reinforcement factor concerned to have 
been detected is raised at the time of study of action and primary reinforcer is 
detected at it whenever a secondary reinforcement factor is detected, the action 
by which the primary reinforcer concerned was detected at least will be 
memorized. 

[0154] The robot equipment with which this invention was applied can be given 
to the action which is going to learn a secondary reinforcement factor as an 



indicator, and can learn action efficiently. 

[0155] In order to attain the purpose mentioned above, moreover, the program 
documentation medium concerning this invention It is the program 
documentation medium by which the action learning program of robot equipment 
which a right hand side is controlled [ learning program ] and makes action 
appear was recorded and in which computer control is possible. The action 
decision process that the action learning program of the above-mentioned robot 
equipment opts for action of 1 based on the appearance inclination of action, 
The above-mentioned right hand side is controlled based on the action for which 
it opted at the above-mentioned action decision process. The primary reinforcer 
detection process of detecting the primary reinforcer beforehand set up from the 
above-mentioned external information which an input means to input external 
information as the action output process of making the action concerned for 
which it opted appearing inputted, The secondary reinforcement factor detection 
process of detecting a secondary reinforcement factor from the 
above-mentioned external information which the above-mentioned input means 
inputted, It has the action storage process of memorizing the action for which it 
opted at the above-mentioned action decision process. At the above-mentioned 
action decision process Whenever the above-mentioned secondary 
reinforcement factor is detected, the appearance inclination of action for the 



secondary reinforcement factor concerned to have been detected is raised. At 
the above-nnentioned action storage process Detection of the above-mentioned 
primary reinforcer records the action learning program of the robot equipment 
characterized by memorizing the action by which the primary reinforcer 
concerned was detected at least. 

[0156] Here, the action by which the secondary reinforcement factor was 
detected is action until it results in the action by which primary reinforcer was 
detected. 

[0157] In the action learning program of the robot equipment currently recorded 
on such a program documentation medium, if the appearance inclination of 
action for the secondary reinforcement factor concerned to have been detected 
is raised at the time of study of action and primary reinforcer is detected at it 
whenever a secondary reinforcement factor is detected, the action by which the 
primary reinforcer concerned was detected at least will be memorized. 
[0158] The robot equipment with which this invention was applied can be given 
to the action which is going to learn a secondary reinforcement factor as an 
indicator, and can learn action efficiently. 



DESCRIPTION OF DRAWINGS 
[Brief Description of the Drawings] 

[Drawing 1] It is the perspective view showing the appearance configuration of 
the robot equipment in the gestalt of operation of this invention. 
[Drawing 2] It is the block diagram showing the circuitry of this robot equipment. 
[Drawing 3] It is the block diagram showing the software configuration of this 
robot equipment. 

[Drawing 4] It is the block diagram showing the configuration of the middleware 

layer in the software configuration of this robot equipment. 

[Drawing 5] It is the block diagram showing the configuration of the application 

layer in the software configuration of this robot equipment. 

[Drawing 6] It is the block diagram showing the configuration of the behavioral 

model library of this application layer. 

[Drawing 7] It is drawing explaining the finite stochastic automaton used as the 
information for the action decision of this robot equipment. 
[Drawing 8] It is drawing showing the state transition table prepared for each 
node of a finite stochastic automaton. 

[Drawing 9] It is the block diagram showing the configuration of the part 
concerning this invention in this robot equipment. 



[Drawing 10] It is a flow chart explaining the procedure in the case of setting up a 
secondary reinforcement factor. 

[Drawing 11] It is a flow chart explaining a series of procedures of the action 
study approach of this robot equipment. 

[Drawing 12] It is drawing explaining the transition during the action which this 
robot equipment performs. 

[Drawing 13] It is drawing explaining a part of action which this robot equipment 
performs. 

[Drawing 14] It is drawing explaining the example of the action instruction 
approach for the conventional animal. 
[Description of Notations] 

1 Robot Equipment, 101 Input Section, 102 Primary Reinforcer Detecting 
Element, 1032nd Reinforcer Associative Section, 104 Action Decision Section, 
105 Action Storage Section, 106 Action Output Section, 107 Speech 
Recognition Section, 108 Associative Storage Section 



